An old PC proved to be a tough challenge for EXO Labs. USB devices were incompatible, forcing a reliance on PS/2 peripherals that needed precise connection order. Rewritable disks failed to be recognized, and a massive 4 TB USB drive exceeded FAT32 limits. This vintage machine, purchased for £118.88, featured a mere 128 MB of RAM, a 1.6 GB hard disk, and ran Windows 98.
To network the system, EXO connected it via Ethernet to an M4 MacBook Pro running FileZilla FTP server software. Assigning static IPs enabled communication with sub-millisecond latency, but executable transfers initially failed until switching the FTP client to binary mode resolved the issue.
After addressing hardware and software hurdles, the Pentium II successfully generated text. A 260,000-parameter Llama-style model achieved 39.31 tokens per second, while a smaller 15-million-parameter model hit 1.03 tokens per second. All inference computations ran solely on the CPU, without GPU involvement.
Using LLAMA2.C Enabled Windows 98 Compatibility
Initial attempts with MinGW were abandoned due to unsupported CMOV instructions. Instead, EXO chose Borland C++ 5.02, an older compiler compatible with Windows 98.
This legacy compiler required code adjustments, such as declaring variables exclusively at the start of functions—before C99 standards introduced more flexibility—and removing or replacing newer language constructs.
The port was developed around Andrej Karpathy’s llama2.c, a compact Llama 2 inference tool implemented as a single C file without external dependencies. Designed primarily for educational purposes rather than full-scale machine learning, it suited the constraints of a vintage OS.
Transforming it for Windows 98 involved replacing long long with a DLONGWORD typedef, adjusting variable declarations, substituting clock_gettime with Windows’ GetTickCount(), and simplifying model loading after direct memory mapping caused segmentation faults. The adapted code was named llama98.c.
Karpathy’s repository includes small TinyStories models that enabled this demo. On an M1 MacBook Air, the 15-million-parameter model runs at about 110 tokens per second, compared to EXO’s Pentium II achieving 1.03 tokens per second. However, these figures originate from differing hardware, not a strict performance comparison.
Llama 3.2 1B Performance Remained a Projection
EXO reported an estimated output speed of 0.0093 tokens per second for Llama 3.2 1B, but the Windows 98 system never ran the full billion-parameter model end-to-end.
This estimate combined testing on a memory-sized model shard plus disk read speed measurements. EXO plans to extend the code with offloading features to eventually run larger models fully.
Memory limits explain the barrier: Karpathy’s standard run.c uses 32-bit floats, consuming four bytes per weight. A 7B parameter float32 Llama 2 checkpoint can require roughly 26 GB of storage.
The project also adopts an int8 quantized version. Karpathy reported that reducing a 7B checkpoint from 26 GB to 6.7 GB sped up performance on Linux from 4.6 to 14 tokens per second, with only a slight quality tradeoff. These details reflect Karpathy’s quantization work, not the Pentium II trial.
BitNet Points Toward Smaller Model Efficiency
As a future direction, EXO discussed BitNet, which compresses model weights to just three levels: -1, 0, and 1, using approximately 1.58 bits per weight.
This ternary scheme changes math operations: multiplication by zero is skipped, by one becomes addition, and by minus one becomes subtraction. This technique reduces storage and computational demands.
EXO suggested a 7-billion-parameter BitNet model needs 1.38 GB of storage, fitting within the old PC’s 1.6 GB disk. They also cited BitNet benchmarks: 52 tokens per second on an M2 Ultra, 18 tokens per second on an Intel i7, and projected 5 to 7 tokens per second for a 100-billion-parameter CPU model. These results derive from BitNet performance studies, not the 1997 machine.
EXO noted releasing MLX-BitNet in April 2024 for Apple Silicon and presented a BitNet variant for protein language modeling at ICML 2024. At the time of their Windows 98 post, no large open BitNet models existed, but training is planned for 2025.
bGPT Model Learned CPU State Prediction in a Simplified Simulated Processor
In a separate February 2024 study, the Beyond Language Models: Byte Models are Digital World Simulators paper explored if models trained on raw bytes could grasp patterns in digital systems like CPU states.
Authors Shangda Wu, Xu Tan, Zili Wang, Rui Wang, Xiaobing Li, and Maosong Sun developed bGPT, testing it with text, images, audio, symbolic music, data format conversion, and CPU state prediction. Their 110-million-parameter model processed files up to 8 KB, segmented into 16-byte patches with a maximum sequence length of 512.
For CPU experiments, they generated 2.1 million training samples from a simplified simulated processor. Each sample contained a 1 KB memory block holding machine instructions, followed by a 16-byte vector representing the CPU register state. The instruction set included 21 types with 43 variations, covering data transfer, logic, and arithmetic operations.
Programs ranged from 1 to 256 random instructions, excluding complex commands like jumps to avoid issues like infinite loops. Each ended with a halt instruction. This setup allowed over 516 million unique possible cases. The test set had an additional 21,000 examples.
Accuracy improved significantly with more training data: at 10,000 examples, bGPT4 reached 27.2404% byte accuracy; at 100,000, bGPT5 achieved 99.9702%; and at one million, bGPT6 hit 99.9961%.
The study observed a sharp performance jump for bGPT5 near its 11th epoch, potentially indicating an emergent skill. However, authors noted this might also stem from non-linear metrics or overfitting.
These findings apply only to the simplified simulated CPU, with authors stressing that real-world modern processors are far more complex.
- Categories:
- News
0 comments
Sign in to Comment