Novalight - very fast tape loading

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Novalight - very fast tape loading

Post by Symoon »

Hi all,

After years working on it, with your help, I'm happy to release:
Novalight, very fast tape loading ;)

You can download it here, compiled for Windows command line:
https://sourceforge.net/projects/euphor ... Novalight/

I hope the C code can be compiled on other platforms.

The work-in-progress thread is here:
viewtopic.php?f=19&t=1360
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

Here's the readme:
Novalight, very fast tape loader for Oric, 02/2019.
By Symoon, based on Fabrice's tools TAP2WAV and TAP2CD.
Version 1.1k

Novalight converts an Oric TAP file into a 44 kHz WAV file that should load very fast on most Orics, ROM 1.0 or ROM 1.1.
It must be used in MS-DOS, or Windows command line (see below).
Default version is compiled in 32 bits and should work on 32/64 bits Windows command line.
Can be compiled in 16-bits mode for old real MS-DOS, and I hope it could be compiled for Unix/Linux.


WARNING:
- requires a *perfect* WAV player (Audacity?).
- do not convert the WAV file, in any way (mp3 or whatever). Each and every sample of the WAV file is important.

In case of failure loading the file:
- keep cellphones away, switch off WiFi
- try different volume settings if loading fails, some Orics require high volume, others very low
- try rebooting your computer playing the WAV if loading fails - this happens with my PC when it's been running for a while!
- some Orics may not load this signal



Usage:
Novalight [ -option1 ] [ -option2 ] <.TAP file> <.WAV file>

Options: -s 'standard speed': use standard speed instead of F16 speed
for the loader (slower, but some Orics may not like F16?)
-o 'old loader': use the ROM 1.1-only loader instead of the
latest 1.0/1.1 loader. Old one has an advantage: it can be
loaded only once, then used with several CALL#100 to load
multiple parts, as long as it remains intact. But no ROM 1.0.
-n 'no loader': generates the Novalight file(s), without loader.
You should load an 'old Novalight' loader yourself first, then
use CALL#100 to load each of the file(s). ROM 1.1 only.
-p 'long pause': generates a 5 seconds silence between each part
of multipart programs (ERE Informatique programs, for instance,
require time to draw a loading screen).



Loading time examples:

Code: Select all

			FAST standard	FAST F16	TAP2CD	Novalight
HIRES screen		0:56		0:34		0:08	0:04 - 0:02
TEXT screen		0:10		0:07		0:05	0:015
Zorgon			4:31		2:37		0:31	0:15
Xenon-1			4:15		2:31s		0:32	0:12
Acheron's Rage		5:12		3.12		0:40	0:12
Oricium			5:45		3:25		0:49	0:18
Lone Raider		5:08		2:59		fails
Trouble In Store	5:14		fails		fails

Programs compatibility:

Code: Select all

		FAST standard	FAST F16	TAP2CD		Novalight
Oric-1		yes		yes		no		yes
Atmos		yes		yes		yes		yes
Emulator	yes		yes		yes		yes
Real tape	yes		no		no		no
Digital player	yes 11kHz	yes 44kHz	yes 22kHz	yes 44kHz

How does it work:
Novalight uses the TAP2CD bit encoding (2 bits per period: 00, 01, 10 and 11), but with shorter periods: 3, 4, 5 and 6 samples for 2 bits, while TAP2CD used the equivalent of 4, 6, 8 and 10 samples. Four periods are required to make a full byte.
Based on statistics made on about 1400 TAP files, showing that Oric files hold 60% of "0" and 40% of "1", Novalight encodes the zeroes on the shortest persiods ("00" is 3 samples long, while on TAP2CD it was "11").
A RLE compression has been added, repeated bytes will be encoded with the shortest possible period (3 samples, 4 samples for the last repeated one).
Two 7-bytes dictionaries are set, stored with the program header and filled with the most repeated bytes that have not been dealed by the RLE compression. Those bytes are also encoded in a shorter way.
Partiy check has been removed.
Stop bits have been merged in the start bit, its length determines the type of encoding that follows (normal byte, RLE compressed, or dictionary encoded)

Novalight's loaded has been minimised: using F16 speed (compatible with standard ROM), inverting bytes to save 10% of loading time (bits 1 coded as 0 and reversed). It loads first a minimal kernel able to load at Novalight speed without compression, then loads at fast speed small banks that do different things: VIA initialization, display, loading of banks, RLE and dictionaries decoding code, VIA restore and progam launch. This reduces the loader's loading time at about 0.6 seconds, plus the banks that follow (very, very short loading time)

Novalight is loaded in page 1, and leaves rather little room for the stack management (54 bytes). Beware if you ever call it in a sub-sub-program or in a loop.

While at it, Novalight also fixes the HIRES CLOAD bug on ROM 1.0: a HIRES file loaded on ROM 1.0 should not be damaged anymore by a black line once loaded (which was the result of the erasing of the displayed of "Loading.." while in HIRES).


Acknowledgements:
Novalight's name is a tribute to Twilighte, the great Oric coder, and Paul Woakes, co-founder of Novagen company, author of Mercenary games and Novaload fast tape loader for C64.
RIP guys, I had the greatest respect for your work.

Thanks to Fabrice for having written the Oric tapes conversion tools long ago, for TAP2CD and for his constant help, giving ideas and answering my questions about tapes, WAVs, interrupts and Oric coding.
Thanks to the Oric community, especially on forums Defence-Force (http://forum.defence-force.org) and Oric.org (http://forums.oric.org). For months, many people here spent time on various coding questions, tests, advice: FredV60, Musepat, Kenneth, Dom50, Voyageur, Oric1-Atmos, Froggy, Godzil, DrPsy, ISS, Chema, Dbug, NekoNoNiaow.
I would *never* have been able to program Novalight without this help.

Enjoy!
User avatar
iss
Wing Commander
Posts: 1641
Joined: Sat Apr 03, 2010 5:43 pm
Location: Bulgaria
Contact:

Re: Novalight - very fast tape loading

Post by iss »

Absolutely incredible work! Congratulation, Symoon!
Thanks for all your effort in pushing CLOAD to the limits.
Releasing sources and the well documented details makes Novalight huge event in this small Oric world!
User avatar
Chema
Game master
Posts: 3014
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Novalight - very fast tape loading

Post by Chema »

Impressive work indeed! A great achievement.

I must find a way to make audio loads again on my Atmos, because I lost the laptop which was really reliable and couldn't find a way to make turbo loads work again :(
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

Thank you guys ;)
@Chema I'm sure you'll find. I'm using an old eeepc for this, those tiny computers are still great for such a job.
And I bet ISS will add it to TapOric at some point anyway - well I'm sure he will, I'm just worried about possible 44/48kHz issues.
User avatar
iss
Wing Commander
Posts: 1641
Joined: Sat Apr 03, 2010 5:43 pm
Location: Bulgaria
Contact:

Re: Novalight - very fast tape loading

Post by iss »

Great piece of software, Symoon, I really can't imagine how many time and effort you put in Novalight but it's worth! Chapeau bas! ;)
The source compiles fine under Linux with GCC (just 2-3 harmless warning - no problem at all) and the tool works nice.

I spotted just one thing which seams like bug:
NL-branch.png
IMO the "+5" should be "+6", right?
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

Ooooh very nice spotting, I think you're right indeed.
Thanks!

It seems I get this bug since the very beginning. Strange, how can it have not crashed when I was running the code on ROM 1.0? does the op code $13 act like a NOP ?

I'm correcting right now and will upload the corrected verion before tonight (train travelling this afternoon)
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

iss wrote: Sun Feb 10, 2019 1:28 pmI spotted just one thing which seams like bug:
NL-branch.png
IMO the "+5" should be "+6", right?
Thanks a lot ISS; it's corrected, tested on Oric-1, and uploaded on SourceForge!

About the time spent on it, I think something like 3 years, with large breaks due to real life. That allows ideas, thinking and so on, to optimize.
The only really boring thing was testing on real machines to find the right working WAV form, and find the right thresholds that would work on most machines.
iss wrote: Sun Feb 10, 2019 1:28 pmThe source compiles fine under Linux with GCC (just 2-3 harmless warning - no problem at all) and the tool works nice.
That's great news!
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

Symoon wrote: Sun Feb 10, 2019 2:19 pmIt seems I get this bug since the very beginning. Strange, how can it have not crashed when I was running the code on ROM 1.0? does the op code $13 act like a NOP ?
I'm puzzled. I've tried a program with $13 $60, called it and it hangs.
But I've loaded again an old test file of Acherons Rage on Oric-1: no problem, though the bug is supposed to be here.
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

Ok I think I got it. Unbelivable luck.
$13: undocumented opcode (2 bytes):
ASO (ab),Y ;13 ab This opcode ASLs the contents of a memory location and then ORs the result with the accumulator.
$63: undocumented opcode (2 bytes):
RRA (ab,X) ;63 ab RRA RORs the contents of a memory location and then ADCs the result with the accumulator.

So 13 20 65 E5 does this:
ASO (20),Y
RRA (E5,X)
instead of calling the "clear status line" code. Which is not visible since this cleaning cleans nothing (Loding from the original CLOAD command is already cleaned, and I removed the display of "Searching" in Novalight) to replace it by "Loading .. PROGRAM NAME".

In conclusion:
- there was a bug indeed
- unbelivable luck made it invisible and not crashing
- I think I could remove this "clear line status" part!
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

By testing heavily again this evening, I found 2 other small problems.
Updated SourceForge.

1- F16 speed was disabled. It's now back so you should see the loader is now much faster to load!
2- I had reduced a pause between two parts of the loader, that worked fine with real machines and emulators, but just found that apparently, it caused problems with multipart programs in Euphoric. Tried with Ere Informatique's Karate, and the 5th or 6th part ended with a syntax error, while it didn't if I put back the original pause. So did I, even though it's a bit unclear why.
User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow »

Good job! I am eager to have a look at the sources to see what magic you had to resort to. ;)
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

I'm just thinking at a possible small optimization for multipart programs.

Once fully loaded, the kernel and common area 0 ($100-$18C) are not destroyed, unless the loaded program decides to use page 1.
So maybe it only needs to be loaded once, then for next parts only loading a reduced version of Common Area 1 is required, without the "load Common area 0" part. And then the other Common areas must be loaded as usual.

This would save I guess around 0.3 or 0.4 second for each additional program. But would require a new option to choose between this or having the full loader every time, which would make the options more complex...
Worth it?
User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow »

How long is the loader (in bytes)?
Could you store it in a darkened part of graphic memory and then wipe it entirely once the loading is done?
User avatar
Symoon
Archivist
Posts: 2307
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Novalight - very fast tape loading

Post by Symoon »

NekoNoNiaow wrote: Tue Feb 12, 2019 1:13 am How long is the loader (in bytes)?
It's a bit complicated to answer, because it's using banks that are loaded as the WAV goes on and crush each other. And they are loading at different speeds: F16 first for a part of the kernel, then a reduced Novalight speed.
So the required room is 202 bytes, but the loader is longer since it swaps data inside this room (the total size is 368 bytes).

Take a look at the XLS file in the Novalight ZIP, it explains how it is built and loaded ;)
NekoNoNiaow wrote: Tue Feb 12, 2019 1:13 am Could you store it in a darkened part of graphic memory and then wipe it entirely once the loading is done?
Page 1 (Fabrice's idea!) seems to be the less used area by programs.
I had thought at choosing an area according to the programs addresses, but it's not enough: once loaded programs can use any RAM area! And there is one or two jumps that are not relative, so relocating it dynamically could be done but with a bit of rework.
Post Reply