Cool stuff! I can see some GPT comments that can be removed
// Increased for better learning
this doesn't tell me anything
// Use the constants from lib.rs
const MAX_SEQ_LEN: usize = 80;
const EMBEDDING_DIM: usize = 128;
const HIDDEN_DIM: usize = 256;
these are already defined in lib.rs, why not use them (as the comment suggests)
leoh 16 hours ago [-]
They should stay, because they are indicative of the fact that this wasn't built with actual understanding.
mitchitized 13 hours ago [-]
You're absolutely correct!
ericdotlee 18 hours ago [-]
Do you think vibe coded rust will rot the quality of language code generally?
6r17 16 hours ago [-]
For AI you definitely need to clean up and I think even targeted learning on some practices would be beneficiary ; for users ; it depends on the people, and I'd argue that vibe-coded rust can be better than just "written-rust" IF the important details and mind of the user are actually focused on what is important ; Eg ; I could vibe-code a lock-free well architect-ed s3 - focus on all the important details that would actually make it high perf - or write some stuff myself 10x slower - which means I will have 10 x less time to work on the important stuff.
However what you asked is wether the vibe coded rust will rot the quality of language ; this is a more difficult to answer to, but I don't think that people who are uninterested in the technics are going to go for rust anyway - from the signals I feedback people are actually not really liking it - they find it too difficult for some reason and prefer to blanket with stuff like C# or python.
Can't explain why.
miki123211 11 hours ago [-]
> I'd argue that vibe-coded rust can be better than just "written-rust
I never thought about it this way, but it actually makes sense. It's just like how Rust / Go / Java / C# can sometimes be orders of magnitude faster than C, only because they're more expressive languages. If you have a limited amount of time, it may be possible to write an efficient, optimal and concurrent algorithm in Java, while in C, all you can do is the simplest possible solution. Linked list versus slices (which are much more cache-friendly) is the perfect example here.
adastra22 18 hours ago [-]
These things will be corrected over time.
yahoozoo 17 hours ago [-]
How do you mean?
tialaramex 19 hours ago [-]
For the constants is it possible the author didn't know how? I remember in my first week of Rust I didn't understand how to name things properly, basically I was overthinking it.
vlovich123 18 hours ago [-]
Lots of signs this is an LLM-generated project. All the emojis in the README are a hint as well.
Oh yea I'm totally running this on my hardware. Extra credit for "from scratch" in the title. The future sucks.
14 hours ago [-]
untrimmed 22 hours ago [-]
As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.
ricardobeat 21 hours ago [-]
Have you tried uv [1]? It has removed 90% of the pain of running python projects for me.
I'm sure it's true and all. But I've been hearing the same claim about all those tools uv is intended to replace, for years now. And every time I try to run any of those, as someone who's not really a python coder, but can shit out scripts in it if needed and sometimes tries to run python software from github, it's been a complete clusterfuck.
So I guess what I'm wondering is, are you a python guy, or are you more like me? because for basically any of these tools, python people tell me "tool X solved all my problems" and people from my own cohort tell me "it doesn't really solve anything, it's still a mess".
If you are one of us, then I'm really listening.
hobofan 19 hours ago [-]
I'm one of you.
I'm about the highest tier of package manager nerd you'll find out there, but despite all that, I've been struggling to create/run/manage venvs out there for ages. Always afraid of installing a pip package or some piece of python-based software (that might muck up Python versions).
I've been semi-friendly with Poetry already, but mostly because it was the best thing around at the time, and a step in the right direction.
uv has truely been a game changer. Try it out!
tinco 18 hours ago [-]
As a Ruby guy: uv makes Python feel like it finally passed the year 2010.
llIIllIIllIIl 18 hours ago [-]
Don’t forget to schedule your colonoscopy as a Ruby guy
Yoric 17 hours ago [-]
As a developer: it basically solved all of my problems that could be solved by a package manager.
As an occasional trainer of scientists: it didn't seem to help my students.
buildbot 17 hours ago [-]
It installs stuff super fast!
It sadly doesn’t solve stuff like transformer_engine being built with cxx11 ABI and pytorch isn’t by default, leading to missing symbols…
OrderlyTiamat 18 hours ago [-]
I'm (reluctantly) a python guy, and uv really is a much different experience for me than all the other tools. I've otherwise had much the same experience as you describe here. Maybe it's because `uv` is built in rust? ¯\_ (ツ)_/¯
But I'd also hesitate to say it "solves all my problems". There's plenty of python problems outside of the core focus of `uv`. For example, I think building a python package for distribution is still awkward and docs are not straightforward (for example, pointing to non-python files which I want to include was fairly annoying to figure out).
beacon294 12 hours ago [-]
It doesn't handle python version management, it only handles pip. It doesn't solve bundling Python.
OoooooooO 17 hours ago [-]
As a mainly Python guy (Data Engineering so new project for every ETL pipeline = a lot of projects) uv solved every problem I had before with pip, conda, miniconda, pipx etc.
J_Shelby_J 19 hours ago [-]
Isn’t UV essentially cargo for python?
adastra22 18 hours ago [-]
Somewhat literally so. It is written in Rust and makes use of the cargo-util crate for some overlapping functionality.
rossant 13 hours ago [-]
I know, but uv truly is different.
jhardy54 19 hours ago [-]
I’m a “Python guy” in that I write Python professionally, but also am like you in that I’ve been extremely underwhelmed by Portry/Pipenv/etc.
Python dependencies are still janky, but uv is a significant improvement over existing tools in both performance and ergonomics.
DiabloD3 21 hours ago [-]
uv is great, but I think the real fix is just abandoning Python.
The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.
trklausss 20 hours ago [-]
Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.
Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language".
If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)
aeve890 20 hours ago [-]
>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.
I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.
DiabloD3 20 hours ago [-]
I am not a huge fan of Go, but if all the world's "serious" Python became Go, the average code quality would skyrocket, so I think I can agree to this proposal.
physicsguy 19 hours ago [-]
Go performance is terrible for numeric stuff though, no SIMD support.
9rx 19 hours ago [-]
That's not really true, but we're talking about a Python replacement for scripting tasks, not core compute tasks, anyway. It is not like Python is the paragon of SIMD support. Any real Python workloads end up being written in C for good reason, using Python only as the glue. Go can also interface with C code, and despite all the flack it gets for its C call overhead it is still significantly faster at calling C code than Python is.
adastra22 18 hours ago [-]
For the record of people reading this, I wrote a multithreaded SIMD-heavy compute task in Go, and it suffered only 5% slowdown vs the original hand-optimized C++ version.
The low level SIMD stuff was called out to over the c FFI bridge; golang was used for the rest of the program.
DiabloD3 19 hours ago [-]
(given the context of LLMs) Unless you're doing CPU-side inference for corner cases where GPU inference is worse, lack of SIMD isn't a huge issue.
There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages.
I think LLVM has it now, its just not super great yet.
wild_egg 19 hours ago [-]
Lots of packages out there using SIMD for lots of things.
You can always drop into straight assembly if you need to as well. Go's assembler DX is quite nice after you get used to it.
pjmlp 15 hours ago [-]
Go itself no, but luckily like in any compiler toolchain, there is an Assembler available.
pclmulqdq 19 hours ago [-]
There are Go SIMD libraries now, and there's also easy use of C libraries via Cgo.
airza 20 hours ago [-]
There's not really another game in town if you want to do fast ML development :/
DiabloD3 20 hours ago [-]
Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum.
Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based.
The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them.
Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust.
mountainriver 19 hours ago [-]
I love Rust and C, I write quite a bit of both. I am an ML engineer by trade.
To say most ML people are using Rust and C couldn’t be further from the truth
Narishma 18 hours ago [-]
They said most people they knew, not most people.
wolvesechoes 16 hours ago [-]
> It, too, would benefit from becoming Rust.
Yet it was created for Python. Someone took that effort and did it. No one took that effort in Rust. End of the story of crab's superiority.
Python community is constantly creating new, great, highly usable packages that become de facto industry standards, and maintain old ones for years, creating tutorials, trainings and docs. Commercial vendors ship Python APIs to their proprietary solutions. Whereas Rust community is going through forums and social media telling them that they should use Rust instead, or that they "cheated" because those libraries are really C/C++ libraries (and BTW those should be done in Rust as well, because safety).
nkozyra 19 hours ago [-]
> Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum.
I wish this were broadly true.
But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling.
I think ML will fade away from Python eventually but right now it's still everywhere.
DiabloD3 17 hours ago [-]
A lot of what I see in ML is all focused around Triton, which is why I mentioned it.
If someone wrote a Triton impl that is all Rust instead, that would do a _lot_ of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all.
airza 19 hours ago [-]
Okay. Humor me.
I want to write a transformer-based classifier for a project. I am accustomed to the pytorch and tensorflow libraries. What is the equivalent using C?
adastra22 17 hours ago [-]
You do know that tensorflow was written in C++ and the Python API bolted on top?
wolvesechoes 17 hours ago [-]
It could be written in mix of Cobol and APL. No one cares.
People saying "oh those Python libraries are just C/C++ libraries with Python API, every language can have them" have one problem - no other language has them (with such extensive documentation, tutorials etc.)
adastra22 16 hours ago [-]
Tensorflow has extensive documentation of its C++ interface, as that is the primary interface for the library (the Python API is a wrapper on top).
wolvesechoes 14 hours ago [-]
I hoped it was quite obvious that by "other languages" I meant "other than Python and C/C++ in which they are written".
At least sibling actually mentioned Java.
adastra22 12 hours ago [-]
Scroll up this thread and the other poster was asking if you can use pytorch and tensorflow from C. Both are C++ libraries, so accessing them from C/C++ is pretty trivial and has first-class support.
wolvesechoes 1 hours ago [-]
You should read more carefully before responding.
I said "beside Python, and C/C++ in which they are written"
You: "you can see people are using it from C".
What a surprise that library usable from Python through wrapped C API has C API!
pjmlp 15 hours ago [-]
PyTorch and Tensorflow also support C++ (naturally) and Java.
airza 14 hours ago [-]
I am. Are you suggesting that as an alternative to the python bindings i should use C to invoke the C++ ABI for tensorflow?
adastra22 12 hours ago [-]
> Okay. Humor me. I want to write a transformer-based classifier for a project. I am accustomed to the pytorch and tensorflow libraries. What is the equivalent using C?
Use C++ bindings in libtorch or tensorflow. If you actually mean C, and not C++, then you would need a shim wrapper. C++ -> C is pretty easy to do.
pjmlp 15 hours ago [-]
PyTorch also supports C++ and Java, Tensorflow also does C++ and Java, Apple AI is exposing ML libraries via Swift, Microsoft is exposing their AI stuff via .NET and Java as well, then there is Julia and Mojo is coming along.
It is happening.
og_kalu 15 hours ago [-]
TensorFlow is a C++ library with a python wrapping, yet nobody (obviously exaggeration) actually uses tensorflow (or torch) in C++ for ML R&D.
It's like people just don't get it. The ML ecosystem in python didn't just spring from the ether. People wanted to interface in python badly, that's why you have all these libraries with substantial code in another language yet development didn't just shift to that language.
If python was fast enough, most would be fine to ditch the C++ backends and have everything in python, but the reverse isn't true. The C++ interface exists, and no-one is using it.
pjmlp 14 hours ago [-]
The existing C++ API is done according to that "beautiful" Google guidelines, thus it could be much better.
However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.
og_kalu 13 hours ago [-]
>However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.
That's not really a reason to think people are using it for that when things like onnxruntime and executorch exist. In fact, they are very likely not using it for that, if only because the torch runtime is too heavy for distribution on the edge anyway (plus android can run python).
Regardless, that's just inference of existing models (which yes I'm sure happens in other languages), not research and/or development of new models (what /u/airza was concerned about), which is probably 99% in python.
pjmlp 3 hours ago [-]
Well, onnxruntime is also having polyglot bindings, and yet another way to avoid Python.
Yes, you can package Python alongside your APK, if you feel like having fun making it compiled with NDK, and running stuff even more slowly in phone ARM chipsets over Dalvik JNI than it already is on desktops.
18 hours ago [-]
pjmlp 19 hours ago [-]
I know Python since version 1.6.
It is great for learning on how to program (BASIC replacement), OS scripting tasks as Perl replacement, and embedded scripting in GUI applications.
Additionally understand PYTHONPATH, and don't mess with anything else.
All the other stuff that is supposed to fix Python issues, I never bothered with them.
Thankfully, other languages are starting to also have bindings to the same C and C++ compute libraries.
wavemode 15 hours ago [-]
Rust is not a viable replacement for Python except in a few domains.
Exuma 20 hours ago [-]
i hate python, but the idea of replacing python with rust is absurd
WhereIsTheTruth 18 hours ago [-]
abandoning Python for Rust in AI would cripple the field, not rescue it
the disease is the cargo cult addiction (which Rust is full of) to micro libraries, not the language that carries 90% of all peer reviewed papers, datasets, and models published in the last decade
every major breakthrough, from AlphaFold to Stable Diffusion, ships with a Python reference implementation because that is the language researchers can read, reproduce, and extend, remove Python and you erase the accumulated, executable knowledge of an entire discipline overnight, enforcing Rust would sabotage the field more than anything
on the topic of uv, it will do more harm than good by enabling and empowering cargo cults on a systemic level
the solution has always been education, teaching juniors to value simplicity, portability and maintainability
stonemetal12 15 hours ago [-]
Nah, it would be like going from chemistry to chemical engineering. Doing chemical reactions in the lab by hand is great for learning but you aren't going to run a fleet of cars on hand made gas. Getting ML out of the lab and into production needs that same mental conversion from CS to SE.
TheAceOfHearts 20 hours ago [-]
Switching to uv made my python experience drastically better.
If something doesn't work or I'm still encountering any kind of error with uv, LLMs have gotten good enough that I can just copy / paste the error and I'm very likely to zero-in on a working solution after a few iterations.
Sometimes it's a bit confusing figuring out how to run open source AI-related python projects, but the combination of uv and iterating on any errors with an LLM has so far been able to resolve all the issues I've experienced.
shepardrtc 18 hours ago [-]
uv has been amazing for me. It just works, and it works fast.
farhanhubble 1 hours ago [-]
I have heard of similar experiences on HN a few times. Haven't seen any such conflicts on real projects in the last five years or so, since I started using Poetry and then UV. I deal with data science code and the people writing it have a tendency to create dependency spaghetti, for example including the Scikit package in a mainly Pytorch code, just because they need a tried-and-tested accuracy() function.
I do remember banging my head against failed dependency resolution in my Early days of Python, circa 2014, with Pip and Conda, etc.
The dependency issues I have faced were mostly due to data science folks pinning exact package versions for the sake of replicability in requirements.txt for example
farhanhubble 1 hours ago [-]
My biggest gripes with Python are:
- exports being broken if code is executed from a different directory
- packaging being more complicated than it should be
and I don't even have too much experience in the area of packaging, besides occasionally publishing to a private repo.
codetiger 21 hours ago [-]
I guess, resource utilization like GPU, etc
Galanwe 20 hours ago [-]
> spent days wrestling with Python dependency hell
I mean I would understand that comment in 2010, but in 2025 it's grossly ridiculous.
virtualritz 15 hours ago [-]
So in 2025, in Python, if I depend on two packages. A and B, and they both depend on different, API-incompatible or behavior-incompatible (or both) versions of C, that won't be an issue?
That's not my experience and e.g. uv hasn't helped me with that. I believe this is an issue with Python itself?
If parent was saying something "grossly ridiculous" I must be doing something wrong too. And I'm happy to hear what as that would lower the pain of using Python.
Well, first, this a purposefully contrived example, that pretty much does not happen in real life scenarios. So you're pretty much acknowledging that there is no real problem by having to resort to such length.
Second, what exactly would you like to happen in that instance? You want to have, in a single project, the same library but at different and conflicting versions. The only way to solve that is to disambiguate, per call site, each use of said library. And guess what, that problem exist and was solved 30 years ago by simply providing different package names for different major version. You want to use both gtk 1 and gtk 2 ? Well you have the "gtk" and "gtk2" package, done, disambiguated. I don't think there is any package manager out there providing "gtk" and having version 1 and 2, it's just "gtk" and "gtk2".
Now we could design a solution around that I guess, nothing is impossible in this brave new world of programing, but that seems like a wasted effort for not-a-problem.
adastra22 13 hours ago [-]
Maybe this doesn’t happen in Python, but I find that hard to believe. This is a common thing in Rust, where cargo does support compiling with multiple versions of the same crate. If I have dependency X that depends on version 1.x of crate Z, and dependency Y which depends on version 2.x, cargo will compile BOTH versions of crate Y, and handle the magic of linking dependencies X and Y to their own, different copies of this common dependency.
steveklabnik 12 hours ago [-]
Yes, Rust can do this. I know Ruby cannot, and I believe Python may not either, but I am less sure about it because I’m less good with Python’s semantics here, but I’d believe your parent.
adastra22 17 hours ago [-]
Yeah, because of a tool written in Rust, copying the Rust way of doing things for Python developers.
Galanwe 16 hours ago [-]
I am not even thinking of `uv`, but rather of pyproject.toml, and the various improvements as to how dependencies are declared and resolved. You don't get much simpler than a toml file listing your dependencies and constraints, along with a lock file.
Also let's keep middle school taunts at home.
zoobab 20 hours ago [-]
"a simple cargo run feels like a dream"
A cargo build that warms up your CPU during winter while recompiling the whole internet is better?
surajrmal 18 hours ago [-]
It has 3 direct dependencies and not too many more transitively. You're certainly not recompiling the internet. If you're going to run a local llm I doubt you're building on a toaster so build speed won't be a big ordeal either.
tracker1 16 hours ago [-]
I recently upped to a 9950X with a gen5 nvme.. TBH, even installing a few programs from cargo (which does compiles) is pretty quick now. Even coming from a 5950X with a gen4 drive.
taminka 22 hours ago [-]
lowkey ppl who praise cargo seem to have no idea of the tradeoffs involved in dependency management
the difficulty of including a dependency should be proportional to the risk you're taking on, meaning it shouldn't be as difficult as it in, say, C where every other library is continually reinventing the same 5 utilities, but also not as easy as it is with npm or cargo, because you get insane dependency clutter, and all the related issues like security, build times, etc
how good a build system isn't equivalent of how easy it is include a dependency, while modern languages should have a consistent build system, but having a centralised package repository that anyone freely pull to/from, and having those dependencies freely take on any number of other dependencies is a bad way to handle dependencies
dev_l1x_be 21 hours ago [-]
> lowkey ppl who praise cargo seem to have no idea
Way to go on insulting people on HN. Cargo is literally the reason why people coming to Rust from languages like C++ where the lack of standardized tooling is giant glaring bomb crater that poses burden on people every single time they need to do some basic things (like for example version upgrades).
i'm saying that ease of dependency inclusion should not be a main criterion for evaluating how good a build system is, not that it isn't the main criterion for many people...
like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?
Sl1mb0 19 hours ago [-]
> dependency inclusion _should not_ be a main criterion for evaluating how good a build system is
That's just like, your opinion, man.
lutusp 17 hours ago [-]
> That's just like, your opinion, man.
I would love to know how many younger readers recognize this classic movie reference.
taminka 19 hours ago [-]
i mean, unless you have some absolute divine truths, that's kind of the best i have :shrug
virtualritz 15 hours ago [-]
There are no truths but your opinion in this case runs counter of what 35 years developing software have taught me.
Obviously, I may be an outlier. Some crank who's just smitten by the proposal of spending his time writing code instead of trying to get a dependency (and its sub-dependencies and their sub-dependencies) to build at all (e.g. C/C++) or to have the right version that works with ALL the code that depends on it (e.g. Python).
I.e. I use cargo foremost (by a large margin) for that reason.
taminka 14 hours ago [-]
in my original comment i specifically mentioned that C (and C++) situation is also too extreme and not optimal...
CodeMage 18 hours ago [-]
Dependency management should most definitely be one of the main criteria for evaluating how good a build system is. What's misguided is intentionally opting for worse dependency management in an attempt to solve a people problem, i.e. being careless about adding dependencies to your project in circumstances when you should be careful.
adwn 19 hours ago [-]
> like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?
I think dev_l1x_be's comment is meant to imply that your believe about people having misguided criteria [for evaluation build systems] is itself misguided, and that your favored approach [that the difficulty of including a dependency should be proportional to the risk you're taking on] is also misguided.
taminka 19 hours ago [-]
my thesis is that negative externalities of build systems are important and i don't know how to convince of importance of externalities someone whose value system is built specifically on ignoring externalities and only factoring in immediate convenience...
huflungdung 21 hours ago [-]
[dead]
quantumspandex 21 hours ago [-]
Security is another problem, and should be tackled systematically. Artificially making dependency inclusion hard is not it and is detrimental to the more casual use cases.
hobofan 18 hours ago [-]
> but having a centralised package repository that anyone freely pull to/from, and having those dependencies freely take on any number of other dependencies is a bad way to handle dependencies
So put a slim layer of enforcement to enact those policies on top? Who's stopping you from doing that?
itsibitzi 21 hours ago [-]
What tool or ecosystem does this well, in your opinion?
taminka 19 hours ago [-]
any language that has a standardised build system (virtually every language nowadays?), but doesn't have a centralised package repository, such that including a dependency is seamless, but takes a bit of time and intent
i like how zig does this, and the creator of odin has a whole talk where he basically uses the same arguments as my original comment to reason why odin doesn't have a package manager
zoobab 18 hours ago [-]
"a standardised build system (virtually every language nowadays?)"
Python packages still manage poorly dependencies that are in another lang like C or C++.
IshKebab 21 hours ago [-]
This is the weirdest excuse for Python's terrible tooling that I've ever heard.
"It's deliberately shit so that people won't use it unless they really have to."
taminka 19 hours ago [-]
i just realised that my comment sounds like it's praising python's package management since it's often so inconvenient to use, i want to mention that that wasn't my intended point, python's package management contains the worst aspects from both words: being centralised AND horrible to use lol
my mistake :)
21 hours ago [-]
MangoToupe 17 hours ago [-]
> the difficulty of including a dependency should be proportional to the risk you're taking on
Why? Dependency hell is an unsolvable problem. Might as well make it easier to evaluate the tradeoff between dependencies and productivity. You can always arbitrarily ban dependencies.
jokethrowaway 21 hours ago [-]
Is your argument that python's package management & ecosystem is bad by design - to increase security?
In my experience it's just bugs and poor decision making on the maintainers (eg. pytorch dropping support for intel mac, leftpad in node) or on the language and package manager developers side (py2->3, commonjs, esm, go not having a package manager, etc).
Cargo has less friction than pypi and npm. npm has less friction than pypi.
And yet, you just need to compromise one lone, unpaid maintainer to wreck the security of the ecosystem.
taminka 19 hours ago [-]
nah python's package management is just straight up terrible by every metric, i just used it as a tangent to talk about how imo ppl incorrectly evaluate build systems
farhanhubble 2 hours ago [-]
I've not written a single line of Rust ever, but I have occasionally looked under the hood of Tensorflow, Pytorch etc. and have been a machine learning practitioner for several years. The succinctness of the interfaces surprised me!
om8 11 hours ago [-]
Have a similar project. Also written in rust, runs in a browser using web assembly
linking both rand-core 0.9.0 and rand-core 0.9.3 which the project could maybe avoid by just specifying 0.9 for its own dep on it
Diggsey 17 hours ago [-]
It doesn't link two versions of `rand-core`. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3".
0xffff2 15 hours ago [-]
So there's no difference at all between "0", "0.9" and "0.9.3" in cargo.toml (Since semver says only major version numbers are breaking)? As a decently experienced Rust developer, that's deeply surprising to me.
What if devs don't do a good job of versioning and there is a real incompatibility between 0.9.3 and 0.9.4? Surely there's some way to actually require an exact version?
Notice how the the minimum bound changes while the upper bound is the same for all of them.
The reason for this is that unless otherwise specified, the ^ operator is used, so "0.9" is actually "^0.9", which then gets translated into the kind of range specifier I showed above.
There are other operators you can use, these are the common ones:
(default) ^ Semver compatible, as described above
>= Inclusive lower bound only
< Exclusive upper bound only
= Exact bound
Note that while an exact bound will force that exact version to be used, it still doesn't allow two semver compatible versions of a crate to exist together. For example. If cargo can't find a single version that satisfies all constraints, it will just error.
For this reason, if you are writing a library, you should in almost all cases stick to regular semver-compatible dependency specifications.
For binaries, it is more common to want exact control over versions and you don't have downstream consumers for whom your exact constraints would be a nightmare.
14 hours ago [-]
steveklabnik 14 hours ago [-]
Note that in the output, there is rand 0.9.0, and two instances of rand_core 0.9.3. You may have thought it selected two versions because you missed the _core there.
> So there's no difference at all between "0", "0.9" and "0.9.3" in cargo.toml
No, there is a difference, in particular, they all specify different minimum bounds.
The trick is that these are using the ^ operator to match, which means that the version "0.9.3" will satisfy all of those constraints, and so Cargo will select 0.9.3 (the latest version at the time I write this comment) as the one version to satisfy all of them.
Cargo will only select multiple versions when it's not compatible, that is, if you had something like "1.0.0" and "0.9.0".
> Surely there's some way to actually require an exact version?
Yes, you'd have to use `=`, like `=0.9.3`. This is heavily discouraged because it would lead to a proliferation of duplication in dependency versions, which aren't necessarily unless you are trying to avoid some sort of specific bugfix. This is sometimes done in applications, but basically should never be done in libraries.
0xffff2 14 hours ago [-]
Sorry, I don't understand the "^ operator" in this context. Do I understand correctly that cargo will basically select the latest release that matches within a major version, so if I have two crates that specify "0.8" and "0.7.1" as dependencies then the compiler will use "0.8.n" for both? And then if I add a new dependency that specifies "0.9.5", all three crates would use "0.9.5"? Assuming I have that right, I'm quite surprised that it works in practice.
steveklabnik 13 hours ago [-]
It’s all good. Let me break it down.
Semver specifies versions. These are the x.y.z (plus other optional stuff) triples you see. Nothing should be complicated there.
Tools that use semver to select versions also define syntax for defining which versions are acceptable. npm calls these “ranges”, cargo calls them “version requirements”, I forget what other tools call them. These are what you actually write in your Cargo.toml or equivalent. These are not defined by the semver specification, but instead, by the tools. They are mostly identical across tools, but not always. Anyway, they often use operators to define the ranges (that’s the name I’m going to use in this post because I think it makes the most sense.) So for example, ‘>3.0.0’ means “any version where x >= 3.” “=3.0.0” means “any version where x is 3, y is 0, and z is 0” which 99% of the time means only one version.
When you write “0.9.3” in a Cargo.toml, you’re writing a range, not a version. When you do not specify an operator, Cargo treats that as if you use the ^ operator. So “0.9.3” is equivalent to “^0.9.3” what does ^ do? It means two things, one if x is 0 and one if x is nonzero. Since “^0.9.3” has x of zero, this range means “any version where x is 0, y is 9, and z is >= 3.” Likewise, “0.9” is equivalent to “^0.9.0” which is “any version where x is 0, y is 9, and z is >=0.”
Putting these two together:
0.9.0 satisfies the latter, but not the former
0.9.1 satisfies the latter, but not the former
0.9.2 satisfies the latter, but not the former
0.9.3 satisfies both
Given that 0.9.3 is a version that has been released, if one package depends on “0.9” and another depends on “0.9.3”, version 0.9.3 satisfies both constraints, and so is selected.
If we had “0.8” and “0.7.1”, no version could satisfy both simultaneously, as “y must be 8” and “y must be 7” would conflict. Cargo would give you both versions in this case, whichever y=8 and y=7 versions have the highest z at the time.
eximius 16 hours ago [-]
This doesn't sound right. If A depends on B and C - B and C can each bring their own versions of D, I thought?
Diggsey 9 hours ago [-]
Within a crate graph, for any given major version of a crate (eg. D v1) only a single minor version can exist. So if B depends on D v1.x, and C depends on D v2.x, then two versions of D will exist. If B depends on Dv1.2 and C depends on Dv1.3, then only Dv1.3 will exist.
I'm over-simplifying a few things here:
1. Semver has special treatment of 0.x versions. For these crates the minor version depends like the major version and the patch version behaves like the minor version. So technically you could have v0.1 and v0.2 of a crate in the same crate graph.
2. I'm assuming all dependencies are specified "the default way", ie. as just a number. When a dependency looks like "1.3", cargo actually treats this as "^1.3", ie. the version must be at least 1.3, but can be any semver compatible version (eg. 1.4). When you specify an exact dependency like "=1.3" instead, the rules above still apply (you still can't have 1.3 and 1.4 in the same crate graph) but cargo will error if no version can be found that satisfies all constraints, instead of just picking a version that's compatible with all dependents.
steveklabnik 14 hours ago [-]
can does not mean must. Cargo attempts to unify (aka deduplicate) dependencies where possible, and in this case, it can find a singular version that satisfies the entire thing.
worldsavior 17 hours ago [-]
This doesn't mean anything. A project can implement things from scratch inefficiently but there might be other libraries the project can use instead of reimplementing.
tonyhart7 22 hours ago [-]
is this satire or does I must know context behind this comment???
stevedonovan 22 hours ago [-]
These are a few well-chosen dependencies for a serious project.
Rust projects can really go bananas on dependencies, partly because it's so easy to include them
obsoleszenz 22 hours ago [-]
The project only has 3 dependencies which i interpret as a sign of quality
leoh 16 hours ago [-]
I don't know if OP intended satire, but either way it is an absurd comment. Think about how "from scratch" this really is.
enricozb 22 hours ago [-]
I did this [0] (gpt in rust) with picogpt, following the great blog by jaykmody [1].
Congrats - there is a very small problem with the LLM - its reusing transformer blocks and you want to use different instances of them.
Its a very cool excercise, I did the same with Zig and MLX a while back, so I can get a nice foundation, but since then as I got hooked and kept adding stuff to it, switched to Pytorch/Transformers.
icemanx 21 hours ago [-]
correction: It's a cool exercise if you write it yourself and not use GPT
Snuggly73 21 hours ago [-]
well, hopefully the author did learn something or at least enjoyed the process :)
(the code looks like a very junior or a non-dev wrote it tbh).
Charon77 22 hours ago [-]
Absolutely love how readable the entire project is
koakuma-chan 21 hours ago [-]
It's AI generated
Revisional_Sin 21 hours ago [-]
How do you know? The over-commenting?
koakuma-chan 21 hours ago [-]
I know because this is how an AI generated project looks. Clearly AI generated README, "clean" code, the way files are named, etc.
magackame 21 hours ago [-]
Not sure myself. Commit messages look pretty human. But the emojis in readme and comments like "// Re-export key structs for easier access", "# Add any test-specific dependencies here if needed" are sus indeed.
cmrdporcupine 21 hours ago [-]
To me it looks like LLM generated README, but not necessarily the source (or at least not all of it).
Or there's been a cleaning pass done over it.
koakuma-chan 21 hours ago [-]
I think pretty clearly the source is also at least partially generated. None the less, just a README like that already sends a strong signal to stop looking and not trust anything written there.
adastra22 17 hours ago [-]
Because the author said so on Reddit.
GardenLetter27 21 hours ago [-]
The repeated Impls are strange.
magackame 21 hours ago [-]
Where? Don't see any on latest main (685467e).
yahoozoo 20 hours ago [-]
`llm.rs` has many `impl LLM` blocks
21 hours ago [-]
emporas 22 hours ago [-]
It is very procedural/object oriented. This is not considered good Rust practice. Iterators make it more functional, which is better, more succinct that is, and enums more algebraic. But it's totally fine for a thought experiment.
15 hours ago [-]
yieldcrv 22 hours ago [-]
Never knew Rust could be that readable. Makes me think other Rust engineers are stuck in a masochistic ego driven contest, which would explain everything else I've encountered about the Rust community and recruiting on that side.
GardenLetter27 21 hours ago [-]
Most Rust code looks like this - only generic library code goes crazy with all the generics and lifetimes, due to the need to avoid unnecessary mallocs and also provide a flexible API to users.
But most people aren't writing libraries.
cmrdporcupine 18 hours ago [-]
Don't underestimate what some programmers trying to prove their cleverness (or just trying to have fun) can do if left unchecked. I think most Rust code does indeed look like this but I've seen plenty of projects that go crazy with lifetimes and generics juggling where they don't have to.
jmaker 22 hours ago [-]
Not sure what you’re alluding to but that’s just ordinary Rust without performance or async IO concerns.
ndai 22 hours ago [-]
I’m curious where you got your training data? I will look myself, but saw this and thought I’d ask. I have a CPU-first, no-backprop architecture that works very well on classification datasets. It can do single‑example incremental updates which might be useful for continuous learning. I made a toy demo to train on tiny.txt and it can predict next characters, but I’ve never tried to make an LLM before. I think my architecture might work well as an on-device assistant or for on-premises needs, but I want to work with it more before I embarrass myself. Any open-source LLM training datasets you would recommend?
huggingface has plenty of openai and antrophic user to assistant chains, beware there are dragons (hallucinations), but good enough for instruction training. I actually recommend distilling kimi k2 instead for instruction following capabilities.
hadlock 15 hours ago [-]
The training data is contained inside main.rs; it looks like about 50 statements about general facts. Probably to keep training time down. If you go off-script things fall apart pretty quickly:
Enter prompt: hello
Model output: What are eclipses ? Assistant : Eclipses occur when one celestial body moves into the shadow of another </s>
Enter prompt: what are facts
Model output: eclipses Assistant : Eclipses Eclipses What What was when What through you ? through you meet through using of What was What tall ? of What thank help you explain using , and do you a patterns you you a patterns through air in tall you help metal metal </s>
Enter prompt: how are mountains formed?
Model output: I ? ' I ' : Of : in happy Hi wind in yeast altering it it </s>
Goto80 22 hours ago [-]
Nice. Mind to put a license on that?
thomask1995 21 hours ago [-]
License added! Good catch
kachapopopow 22 hours ago [-]
This looks rather similar to when I asked an AI to implement a basic xor problem solver I guess fundementally there's really only a very limited amount of ways to implement this.
selinkocalar 10 hours ago [-]
The memory safety guarantees in Rust are probably useful here given how easy it is to have buffer overflows in transformer implementations. CUDA kernels are still going to dominate performance though.
Curious about the tokenization approach - are you implementing BPE from scratch too or using an existing library?
abricq 21 hours ago [-]
This is great ! Congratulations. I really like your project, especially I like how easily it is to peak at.
Do you plan on moving forward with this project ? I seem to understand that all the training is done on the CPU, and that you have next steps regarding optimizing that. Do you consider GPU accelerations ?
Also, do you have any benchmarks on known hardware ? Eg, how long would it take to train on a macbook latest gen or your own computer ?
thomask1995 14 hours ago [-]
HI! OG Author here.
Honestly, I don't know.
This was purely a toy project/thought experiment to challenge myself to learn exactly how these LLMs worked.
It was super cool to see the loss go down and it actually "train".
This is SUPER far from a the real deal. Maybe it could be cool to see how far a fully in memory LLM running on CPU can go.
capestart 20 hours ago [-]
Very cool project, always nice to see deep learning built from scratch in Rust without heavy frameworks.
chcardoz 12 hours ago [-]
super fun!! I am running it right now and going to use it to train on a corpus of my own writing to make a gpt of myself.
yobbo 16 hours ago [-]
Very nice! Next thing to add would be numerical gradient testing.
tripplyons 15 hours ago [-]
Is that where you approximate a partial derivative as a difference in loss over a small difference in a single parameter's value?
Seems like a great way to verify results, but it has the same downsides as forward mode automatic differentiation since it works in a pretty similar fashion.
yobbo 15 hours ago [-]
Yes, the purpose is to verify the gradient computations which are typically incorrect on the first try for things like self-attention and softmax. It is very slow.
It is not necessary for auto-differentiation, but this project does not use that.
ericdotlee 18 hours ago [-]
This is incredibly cool, but I wonder when more of the AI ecosystem will move past python tooling into something more... performant?
Very interesting to already see rust based inference frameworks as well.
leoh 16 hours ago [-]
"Python" is perfectly performant for AI and this demonstrates a deep lack of understanding. Virtually every library in python used for AI delegates to lower-level code written in C++.
tcfhgj 14 hours ago [-]
well, not all the time, e.g. orchestration and handling between multiple libraries
bionhoward 18 hours ago [-]
That time to first token is impressive, it seems like it responds immediately
lutusp 18 hours ago [-]
It would have been nice to see a Rust/Python time comparison for both development and execution. You know, the "bottom line"?
// Increased for better learning
this doesn't tell me anything
// Use the constants from lib.rs
const MAX_SEQ_LEN: usize = 80;
const EMBEDDING_DIM: usize = 128;
const HIDDEN_DIM: usize = 256;
these are already defined in lib.rs, why not use them (as the comment suggests)
However what you asked is wether the vibe coded rust will rot the quality of language ; this is a more difficult to answer to, but I don't think that people who are uninterested in the technics are going to go for rust anyway - from the signals I feedback people are actually not really liking it - they find it too difficult for some reason and prefer to blanket with stuff like C# or python.
Can't explain why.
I never thought about it this way, but it actually makes sense. It's just like how Rust / Go / Java / C# can sometimes be orders of magnitude faster than C, only because they're more expressive languages. If you have a limited amount of time, it may be possible to write an efficient, optimal and concurrent algorithm in Java, while in C, all you can do is the simplest possible solution. Linked list versus slices (which are much more cache-friendly) is the perfect example here.
https://old.reddit.com/r/rust/comments/1nguv1a/i_built_an_ll...
[1] https://github.com/astral-sh/uv
So I guess what I'm wondering is, are you a python guy, or are you more like me? because for basically any of these tools, python people tell me "tool X solved all my problems" and people from my own cohort tell me "it doesn't really solve anything, it's still a mess".
If you are one of us, then I'm really listening.
I'm about the highest tier of package manager nerd you'll find out there, but despite all that, I've been struggling to create/run/manage venvs out there for ages. Always afraid of installing a pip package or some piece of python-based software (that might muck up Python versions).
I've been semi-friendly with Poetry already, but mostly because it was the best thing around at the time, and a step in the right direction.
uv has truely been a game changer. Try it out!
As an occasional trainer of scientists: it didn't seem to help my students.
It sadly doesn’t solve stuff like transformer_engine being built with cxx11 ABI and pytorch isn’t by default, leading to missing symbols…
But I'd also hesitate to say it "solves all my problems". There's plenty of python problems outside of the core focus of `uv`. For example, I think building a python package for distribution is still awkward and docs are not straightforward (for example, pointing to non-python files which I want to include was fairly annoying to figure out).
Python dependencies are still janky, but uv is a significant improvement over existing tools in both performance and ergonomics.
The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.
Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language".
If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)
I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.
The low level SIMD stuff was called out to over the c FFI bridge; golang was used for the rest of the program.
There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages.
I think LLVM has it now, its just not super great yet.
You can always drop into straight assembly if you need to as well. Go's assembler DX is quite nice after you get used to it.
Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based.
The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them.
Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust.
To say most ML people are using Rust and C couldn’t be further from the truth
Yet it was created for Python. Someone took that effort and did it. No one took that effort in Rust. End of the story of crab's superiority.
Python community is constantly creating new, great, highly usable packages that become de facto industry standards, and maintain old ones for years, creating tutorials, trainings and docs. Commercial vendors ship Python APIs to their proprietary solutions. Whereas Rust community is going through forums and social media telling them that they should use Rust instead, or that they "cheated" because those libraries are really C/C++ libraries (and BTW those should be done in Rust as well, because safety).
I wish this were broadly true.
But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling.
I think ML will fade away from Python eventually but right now it's still everywhere.
If someone wrote a Triton impl that is all Rust instead, that would do a _lot_ of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all.
People saying "oh those Python libraries are just C/C++ libraries with Python API, every language can have them" have one problem - no other language has them (with such extensive documentation, tutorials etc.)
At least sibling actually mentioned Java.
I said "beside Python, and C/C++ in which they are written"
You: "you can see people are using it from C".
What a surprise that library usable from Python through wrapped C API has C API!
Use C++ bindings in libtorch or tensorflow. If you actually mean C, and not C++, then you would need a shim wrapper. C++ -> C is pretty easy to do.
It is happening.
It's like people just don't get it. The ML ecosystem in python didn't just spring from the ether. People wanted to interface in python badly, that's why you have all these libraries with substantial code in another language yet development didn't just shift to that language.
If python was fast enough, most would be fine to ditch the C++ backends and have everything in python, but the reverse isn't true. The C++ interface exists, and no-one is using it.
However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.
That's not really a reason to think people are using it for that when things like onnxruntime and executorch exist. In fact, they are very likely not using it for that, if only because the torch runtime is too heavy for distribution on the edge anyway (plus android can run python).
Regardless, that's just inference of existing models (which yes I'm sure happens in other languages), not research and/or development of new models (what /u/airza was concerned about), which is probably 99% in python.
Yes, you can package Python alongside your APK, if you feel like having fun making it compiled with NDK, and running stuff even more slowly in phone ARM chipsets over Dalvik JNI than it already is on desktops.
It is great for learning on how to program (BASIC replacement), OS scripting tasks as Perl replacement, and embedded scripting in GUI applications.
Additionally understand PYTHONPATH, and don't mess with anything else.
All the other stuff that is supposed to fix Python issues, I never bothered with them.
Thankfully, other languages are starting to also have bindings to the same C and C++ compute libraries.
the disease is the cargo cult addiction (which Rust is full of) to micro libraries, not the language that carries 90% of all peer reviewed papers, datasets, and models published in the last decade
every major breakthrough, from AlphaFold to Stable Diffusion, ships with a Python reference implementation because that is the language researchers can read, reproduce, and extend, remove Python and you erase the accumulated, executable knowledge of an entire discipline overnight, enforcing Rust would sabotage the field more than anything
on the topic of uv, it will do more harm than good by enabling and empowering cargo cults on a systemic level
the solution has always been education, teaching juniors to value simplicity, portability and maintainability
If something doesn't work or I'm still encountering any kind of error with uv, LLMs have gotten good enough that I can just copy / paste the error and I'm very likely to zero-in on a working solution after a few iterations.
Sometimes it's a bit confusing figuring out how to run open source AI-related python projects, but the combination of uv and iterating on any errors with an LLM has so far been able to resolve all the issues I've experienced.
I do remember banging my head against failed dependency resolution in my Early days of Python, circa 2014, with Pip and Conda, etc.
The dependency issues I have faced were mostly due to data science folks pinning exact package versions for the sake of replicability in requirements.txt for example
- exports being broken if code is executed from a different directory
- packaging being more complicated than it should be
and I don't even have too much experience in the area of packaging, besides occasionally publishing to a private repo.
I mean I would understand that comment in 2010, but in 2025 it's grossly ridiculous.
That's not my experience and e.g. uv hasn't helped me with that. I believe this is an issue with Python itself?
If parent was saying something "grossly ridiculous" I must be doing something wrong too. And I'm happy to hear what as that would lower the pain of using Python.
I.e. this was assumably true three years ago:
https://stackoverflow.com/questions/70828570/what-if-two-pyt...
Second, what exactly would you like to happen in that instance? You want to have, in a single project, the same library but at different and conflicting versions. The only way to solve that is to disambiguate, per call site, each use of said library. And guess what, that problem exist and was solved 30 years ago by simply providing different package names for different major version. You want to use both gtk 1 and gtk 2 ? Well you have the "gtk" and "gtk2" package, done, disambiguated. I don't think there is any package manager out there providing "gtk" and having version 1 and 2, it's just "gtk" and "gtk2".
Now we could design a solution around that I guess, nothing is impossible in this brave new world of programing, but that seems like a wasted effort for not-a-problem.
Also let's keep middle school taunts at home.
A cargo build that warms up your CPU during winter while recompiling the whole internet is better?
the difficulty of including a dependency should be proportional to the risk you're taking on, meaning it shouldn't be as difficult as it in, say, C where every other library is continually reinventing the same 5 utilities, but also not as easy as it is with npm or cargo, because you get insane dependency clutter, and all the related issues like security, build times, etc
how good a build system isn't equivalent of how easy it is include a dependency, while modern languages should have a consistent build system, but having a centralised package repository that anyone freely pull to/from, and having those dependencies freely take on any number of other dependencies is a bad way to handle dependencies
Way to go on insulting people on HN. Cargo is literally the reason why people coming to Rust from languages like C++ where the lack of standardized tooling is giant glaring bomb crater that poses burden on people every single time they need to do some basic things (like for example version upgrades).
Example:
https://github.com/facebook/folly/blob/main/build.sh
like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?
That's just like, your opinion, man.
I would love to know how many younger readers recognize this classic movie reference.
Obviously, I may be an outlier. Some crank who's just smitten by the proposal of spending his time writing code instead of trying to get a dependency (and its sub-dependencies and their sub-dependencies) to build at all (e.g. C/C++) or to have the right version that works with ALL the code that depends on it (e.g. Python).
I.e. I use cargo foremost (by a large margin) for that reason.
I think dev_l1x_be's comment is meant to imply that your believe about people having misguided criteria [for evaluation build systems] is itself misguided, and that your favored approach [that the difficulty of including a dependency should be proportional to the risk you're taking on] is also misguided.
So put a slim layer of enforcement to enact those policies on top? Who's stopping you from doing that?
i like how zig does this, and the creator of odin has a whole talk where he basically uses the same arguments as my original comment to reason why odin doesn't have a package manager
Python packages still manage poorly dependencies that are in another lang like C or C++.
"It's deliberately shit so that people won't use it unless they really have to."
my mistake :)
Why? Dependency hell is an unsolvable problem. Might as well make it easier to evaluate the tradeoff between dependencies and productivity. You can always arbitrarily ban dependencies.
In my experience it's just bugs and poor decision making on the maintainers (eg. pytorch dropping support for intel mac, leftpad in node) or on the language and package manager developers side (py2->3, commonjs, esm, go not having a package manager, etc).
Cargo has less friction than pypi and npm. npm has less friction than pypi.
And yet, you just need to compromise one lone, unpaid maintainer to wreck the security of the ecosystem.
In-browser demo: https://galqiwi.github.io/aqlm-rs
Source code: https://github.com/galqiwi/demo-aqlm-rs
Looking good!
yep, still looks relatively good.
What if devs don't do a good job of versioning and there is a real incompatibility between 0.9.3 and 0.9.4? Surely there's some way to actually require an exact version?
The reason for this is that unless otherwise specified, the ^ operator is used, so "0.9" is actually "^0.9", which then gets translated into the kind of range specifier I showed above.
There are other operators you can use, these are the common ones:
Note that while an exact bound will force that exact version to be used, it still doesn't allow two semver compatible versions of a crate to exist together. For example. If cargo can't find a single version that satisfies all constraints, it will just error.For this reason, if you are writing a library, you should in almost all cases stick to regular semver-compatible dependency specifications.
For binaries, it is more common to want exact control over versions and you don't have downstream consumers for whom your exact constraints would be a nightmare.
> So there's no difference at all between "0", "0.9" and "0.9.3" in cargo.toml
No, there is a difference, in particular, they all specify different minimum bounds.
The trick is that these are using the ^ operator to match, which means that the version "0.9.3" will satisfy all of those constraints, and so Cargo will select 0.9.3 (the latest version at the time I write this comment) as the one version to satisfy all of them.
Cargo will only select multiple versions when it's not compatible, that is, if you had something like "1.0.0" and "0.9.0".
> Surely there's some way to actually require an exact version?
Yes, you'd have to use `=`, like `=0.9.3`. This is heavily discouraged because it would lead to a proliferation of duplication in dependency versions, which aren't necessarily unless you are trying to avoid some sort of specific bugfix. This is sometimes done in applications, but basically should never be done in libraries.
Semver specifies versions. These are the x.y.z (plus other optional stuff) triples you see. Nothing should be complicated there.
Tools that use semver to select versions also define syntax for defining which versions are acceptable. npm calls these “ranges”, cargo calls them “version requirements”, I forget what other tools call them. These are what you actually write in your Cargo.toml or equivalent. These are not defined by the semver specification, but instead, by the tools. They are mostly identical across tools, but not always. Anyway, they often use operators to define the ranges (that’s the name I’m going to use in this post because I think it makes the most sense.) So for example, ‘>3.0.0’ means “any version where x >= 3.” “=3.0.0” means “any version where x is 3, y is 0, and z is 0” which 99% of the time means only one version.
When you write “0.9.3” in a Cargo.toml, you’re writing a range, not a version. When you do not specify an operator, Cargo treats that as if you use the ^ operator. So “0.9.3” is equivalent to “^0.9.3” what does ^ do? It means two things, one if x is 0 and one if x is nonzero. Since “^0.9.3” has x of zero, this range means “any version where x is 0, y is 9, and z is >= 3.” Likewise, “0.9” is equivalent to “^0.9.0” which is “any version where x is 0, y is 9, and z is >=0.”
Putting these two together:
Given that 0.9.3 is a version that has been released, if one package depends on “0.9” and another depends on “0.9.3”, version 0.9.3 satisfies both constraints, and so is selected.If we had “0.8” and “0.7.1”, no version could satisfy both simultaneously, as “y must be 8” and “y must be 7” would conflict. Cargo would give you both versions in this case, whichever y=8 and y=7 versions have the highest z at the time.
I'm over-simplifying a few things here:
1. Semver has special treatment of 0.x versions. For these crates the minor version depends like the major version and the patch version behaves like the minor version. So technically you could have v0.1 and v0.2 of a crate in the same crate graph.
2. I'm assuming all dependencies are specified "the default way", ie. as just a number. When a dependency looks like "1.3", cargo actually treats this as "^1.3", ie. the version must be at least 1.3, but can be any semver compatible version (eg. 1.4). When you specify an exact dependency like "=1.3" instead, the rules above still apply (you still can't have 1.3 and 1.4 in the same crate graph) but cargo will error if no version can be found that satisfies all constraints, instead of just picking a version that's compatible with all dependents.
Rust projects can really go bananas on dependencies, partly because it's so easy to include them
[0]: https://github.com/enricozb/picogpt-rust [1]: https://jaykmody.com/blog/gpt-from-scratch/
Its a very cool excercise, I did the same with Zig and MLX a while back, so I can get a nice foundation, but since then as I got hooked and kept adding stuff to it, switched to Pytorch/Transformers.
(the code looks like a very junior or a non-dev wrote it tbh).
Or there's been a cleaning pass done over it.
But most people aren't writing libraries.
For just plain text, I really like this one - https://huggingface.co/datasets/roneneldan/TinyStories
Enter prompt: hello Model output: What are eclipses ? Assistant : Eclipses occur when one celestial body moves into the shadow of another </s>
Enter prompt: what are facts Model output: eclipses Assistant : Eclipses Eclipses What What was when What through you ? through you meet through using of What was What tall ? of What thank help you explain using , and do you a patterns you you a patterns through air in tall you help metal metal </s>
Enter prompt: how are mountains formed? Model output: I ? ' I ' : Of : in happy Hi wind in yeast altering it it </s>
Do you plan on moving forward with this project ? I seem to understand that all the training is done on the CPU, and that you have next steps regarding optimizing that. Do you consider GPU accelerations ?
Also, do you have any benchmarks on known hardware ? Eg, how long would it take to train on a macbook latest gen or your own computer ?
Honestly, I don't know.
This was purely a toy project/thought experiment to challenge myself to learn exactly how these LLMs worked.
It was super cool to see the loss go down and it actually "train".
This is SUPER far from a the real deal. Maybe it could be cool to see how far a fully in memory LLM running on CPU can go.
Seems like a great way to verify results, but it has the same downsides as forward mode automatic differentiation since it works in a pretty similar fashion.
It is not necessary for auto-differentiation, but this project does not use that.
Very interesting to already see rust based inference frameworks as well.