NTFS, fat32, exfat, could I theoretically create my own filesystem? If so would my computer even be able to work with most files or connect to other devices?
You can! All of those filesystems you mentioned were created by people, and you can do it just the same way. It is, however, quite a lot of work to get something as good as any of those, let alone better. You’ve also hit on one of the problems - nothing else is going to support your filesystem.
If you’re interested in trying anyway, just out of curiosity, do a little research into FUSE, Filesystem in Userspace. FUSE is a tool that lets you write a filesystem without needing to integrate with the very low-level parts of the operating system, which takes some burden off of your implementation.
Google fails me but my favorite application for FUSE was storing files in headers of ICMP packets using network latency for persistence.
I wonder if anyone has ever passed messages between spacecraft as a peculiar form of delay line memory – or pinged a satellite at a predictable distance as part of a timing system…
No but I look forward to this in the next Andy Weir novel
or pinged a satellite at a predictable distance as part of a timing system…
Isn’t that just GPS in reverse? I mean, same equation, different dependent variable 😁
I vaguely recall what was effectively a delay-line implemented using terrahertz-modulated lasers aimed at the retro reflector on the moon from Earth’s surface. The data storage capacity was something in either the hundreds of GBs or low TBs. But I can’t find the reference.
I think I’ve read about a similar project where someone would send packages across the internet, and use the delay as a form of memory. The capacity wasn’t great, but the idea was really cool regardless.
The harder drive! https://youtube.com/watch?v=JcJSW7Rprio
I love modern art
So a network version of an acoustic delay line?
This is why I love Lemmy, I can ask this obscure, unrealistic question, and people will still answer it with the “yeah you probably shouldn’t but here’s how you could”
Look up reiserfs.
You can do anything if you put your mind to it.
Like murder your estranged wife and watch support for your FS collapse, right?
Did I stutter?
That’s how those filesystems came into existence: someone designed them.
Yes, you can write your own filesystem, and use it on your own drives.
Yes, you will continue to be able to use other filesystems, unless you intentionally remove them from the system.
Building a filesystem essentially means linking a directory of filenames to physical blocks and handling CRUD operations. It’s not that hard. The hard part comes when you go beyond the basics to build something efficient with useful features. For example, fast access, journaling and fragmentation are all challenging topics. You can try without messing with the kernel by creating an in-memory filesystem (essentially a block of RAM) and playing with the I/O.
Lots of good answers, especially using FUSE for experimentation. One thing I’ll add is that if you just didn’t want to use any filesystem at all … you don’t have to!
At least in the Unix realm, a disk drive is just a bunch of contiguous blocks, and you can put whatever you want in them. Of course, Unix itself famously needs a filesystem for itself, but if you want to just store all your giant binary blobs – cryptocurrency block chain? – directly onto a drive without the pesky overhead or conveniences of a filesystem, that’s doable.
It’s not generally a useful idea to treat a disk drive as though it’s a tape drive, but it does work. And going further into that analogy, you can use “tar” to collect multiple files and fit them onto the drive, since a tarball preserves file metadata and the borders between files, but not much else. This is the original use of tar – “tape archive” – for storing Unix files onto tape, because the thought of using tape as working storage with a filesystem was – and still is – a terrible idea. And that’s basically the original impetus for a filesystem: it’s better than linear access media.
Yeah you could. Noone else
couldwould bother to use your usb/hdd/ssd with your file system unless you gave them the drivers.NTFS, fat32, exfat, could I theoretically create my own filesystem?
Yes. There are many different file systems and you can absolutely create your own. Making one that is reliable and performs well, and/or is something you can actually use for the disk that you boot from, generally involves low-level kernel programming and years of work - and is not exactly a beginner’s programming project.
However, you can also more easily play with implementing filesystems in a high-level language using FUSE.
If so would my computer even be able to work with most files or connect to other devices?
Your computer can use many different filesystems at the same time. You can also store a filesystem in a file on another filesystem, rather than dedicating a partition of a physical disk to it. So, yes, you can use a filesystem of your own design at the same time you are using other storage devices formatted with more common filesystems.
I mean, short answer is… yes…
But, eh, like. That’s a technicality. Chances are the task is something beyond typical individuals these days.
But we’re on Lemmy, and that comes with a huge selection bias. In fact, you don’t really find that many typical individuals here. A significant part of the population here uses Linux as a hobby, or does something technical for a living. Possibly both!
I really don’t think it’s that far fetched to imagine that it’s possible to find someone in here who has the time, energy, enthusiasm, patience knowledge and intelligence to build a new file system. I’m pretty sure you can also find more than one person capable of writing their own drivers or libraries.
sure. But it’s a task that usually takes entire teams of highly competent and experienced people. Computer science, software engineering. Database management, lots of math.
Like ext4 took contributions from several orgs and numerous individuals over the course of years to develop. if a single person can do it, it’s the work of a lifetime.
A guy is currently doing this with bcachefs.
He’s also making life very hard for himself by being arrogant and terrible at communication, but that’s a different story.
He’s also making life very hard for himself by being arrogant and terrible at communication
Isn’t he just trying to do things his ways?
I wish to advocate in the name of DIY minimalism. That is to say, it’s true that none of us – Linus Torvalds is not in the room, right? – can hope to churn out anything approaching a full-blown filesystem on the order of ext4 or NTFS if we worked our entire lives. But if those filesystems were the end-all-be-all of innovation in those spaces, the richness and intrigue of computer science would have died out long ago, relegated to only the pinnacle of engineers and no one else.
But I feel like that can’t quite be the case, because all engineering is about achieving careful balances. And as fine as ext4 is, it must be said that it’s anything but minimal. It’s full-featured, which also implies that it might have more than what any one person requires. If OP wants to write a very compact filesystem designed for 8-bit microprocessors, I can’t badger them with ext4’s existence, because that’s not going to be usable on an 8 bit machine.
Much like how Python includes a really tiny HTTP server, and we can all agree that it’s orders of magnitude less sophisticated than nginx, such implementations can have their time in the sun. And I think a tiny, absurdly minimal, almost code-golf of a filesystem, might have a place in this world, if OP really wants to undertake that effort.
Computer science, I wish to believe, still has doors awaiting exploration.
Computer science, I wish to believe, still has doors awaiting exploration.
There was a time where you could grab a walking stick, a cloak, and a pack and go see things no other human has ever recorded seeing; and then that exploration was done and we needed boats or whatever vehicle to get there. Then the boats had to get bigger, and had to be powered by wind rather than oar, then we had to go down, into the depths of the oceans, or up into the vastness of space. We still haven’t explored all of the ocean, seen everything this world has to see.
with each new horizon, there is a new place to explore. a new richness to discover.
This is true, also, of computer science; and all sorts of other frontiers.
I’m not trying to disuade the OP or anyone else. Even if they’re never even close to successful; it’s their time to spend. if they enjoy the work; then by all means. It’s more of a warning… it’s not the kind of thing that’s going to be a weekend project. (“There be dragons,”)
Reminds me of something a coworker once told me. If you had a time machine and went back to the year 1825, there would be an absurd amount of basic chemistry you could discover. Some of it doesn’t even require any fancy equipment, but if you had a proper lab at your disposal, you could become famous in no time.
Making something comparable with ext4 would probably take an autistic genius who dedicates their every waking moment to this project. Someone who eats, drinks and breathes numbers.
Other people can hope to make a fun little project that teaches them about file systems and storage hardware. Might not be a competitor to anything, but it doesn’t have to be. People like to tinker with all sorts of weird stuff, like installing doom on a toaster or something. The way I see it, this FS project is probably in that category.
Yeah, but that’s like answering “Oreos, Jaffa Cakes, Biscoff, could I theoretically bake my own cookies?” with “It’s a task that usually takes entire teams of highly competent and experienced people. Food scientists, industrial designers, supply chain management, factory design, lots of math”
Well, yes, but OP is not trying to make a product to compete with these examples, but clearly just to learn a bit about how computers work. The answer is absolutely yes – to both cookies and file systems – and they can both be done in one afternoon.
If you just want to do it for the fuck of it look at how old simple filesystems like FAT12 work and implement them in whatever high-level programming language you’re most comfortable, maybe with FUSE as others have suggested since there are beginner-level tutorials for that.
Yes, but it would probably be a multi-decade, career-defining project if you want it to be any good. It would likely work with all files since it’s all just data once you get to the FS level anyway, but other devices would need an appropriate driver (that you would also need to write) in order to use it.
Note that I’m not an expert in this field, so others should correct me if I’ve got something wrong here.
Also, once you finish your filesystem, don’t murder your wife or otherwise people will stop using it.
Oh don’t worry, knowing me, I would create it, use it for a single drive and never touch it again. Its the fun of creation that counts. Also, I quite like my wife.
Doesn’t have to be world changing or even practical. A project like that would still teach you a ton about storage hardware, how file systems work, programming, mathematics etc. Some of these lessons could even be useful, but above all, a project like this should be fun and interesting.
Why do you think people install Linux and run Doom on all the weirdest hardware they can find. This is the spirit that drives innovation.
Don’t let expectations hold you back. Make your own FS, and have fun.
I do think it would be fun, I would also love to just have a bit of tech that is “all my own” in a way
Besides, having your own FS is pretty bad ass in the same way as running Linux From Scratch.
FYI you might be interested in WinBTRFS:
https://github.com/maharmstone/btrfs
Which is basically what you are describing: shoehorning support for a new file system into Windows.
Linux in particular has all sorts of exotic and special purpose filesystem, like swap on spare GPU VRAM, software managed SSD stacks and such.
There’s also ways to support ext3-4 on windows as well. And FUSE is also very interesting as it shows you what kind of esoteric stuff can be mapped onto a fake file system.
A guy on YouTube literally made a physical floppy disk from scratch so I dont see why not.
Someone make the Lolcats programming language for a laugh.
Someone make the Lolcats programming language for a laugh.
Rabbit hole! There’s a whole genre of esoteric languages; Malbolge was mentioned by Sherlock Holmes even (not the original one of course).
People have and do, but the effort is ridiculous and requires some very high-level computer science or computer engineering skills.
Creating a drive format is the easy part; getting it generally supported is the hardest part.
