How big is your media library?
-
Ah yes. My storage system is 2 x Supermicro CSE-846 cases. Only one has a CPU and motherboard, the other is acting as a plain Jane JBOD.
Hard drives I have 21 x 8TB 7200RPM mix of Seagate and Western Digital and 4 x 16TB 7200RPM from Seagate. I use mergerfs and snapraid. Mergerfs presents all the 21 8TB drives as one mount point. Snapraid uses the 4 16TB drives to provide 4 parity drives. Note that snapraid is not live and the parity is only updated after running a "snapraid sync" which I run nightly.
I only backup my songs and music videos. The rest is easy to get again. I have a script that generates a list of every single file I have each night. So if the day comes it wouldn't take too long to get back to where I was. The other reason I use mergerfs is if 1 drive dies, I only lose the files on that one drive and not the entire array. The truely important stuff such as tax documents, mortgage details, family pictures, will & estate documents are stored on a 2 x 8TB RAID1 and all backed up nice a safe using Proxmox PBS. The PBS datastore is synced to 2 remote locations as well as to external drives that I keep offline and rotate.
Nice write-up. I thought I had a large library (24TB) and my off site backup is starting to get full. I backup everything though but I have long debated on if there's a point of keeping movies and TV since they'll likely always be available. Anyway, I never thought of generating a list of files and eliminating the stuff that's not particularly important. Good idea.
-
OK Netflix, you don't count
Haha. Thanks. I really didn't want to pay Netflix or any other streaming service. But it might have been cheaper than hdds and electricity.
This is something I've been building for over 10 years at this point. I've gone through so many iterations of servers and storage architecture. I've lost my entire TV and movie library multiple times. (I don't back it up because a. It's expensive at this scale and b. this data is easy to rebuild over time.)
It's been a part of learning about hosting and data management that I've brought to/from my work.
-
Background: I've been writing a new media server like Jellyfin or Plex, and I'm thinking about releasing it as an OSS project. It's working really well for me already, so I've started polishing up the install process, writing getting started docs, stuff like that.
I'm interested in how other folks have set up their media libraries. Especially the technical details around how files are encoded and organized.
My media library currently has about 1,100 movies and just shy of 200 TV shows. I've encoded everything as high quality AV1 video with Opus audio, in a WebM container. Subtitles and chapters are in a separate WebVTT file alongside the video. The whole thing is currently about 9TB. With few exceptions, I sourced everything directly from Blu-ray or DVD using MakeMKV. It's organized pretty close to how Jellyfin wants it.
What about you?
12.8TB. Mostly uncompressed rips from Blu-rays, some DVDs, some from iTunes Store. Some from the high seas, but not in a long time because the market solved that problem with streaming.
-
I went with a GeForce RTX 4060. Cost was about $300.
Cool, thanks!
-
I thought 20TB of storage would last me forever
I've had low storage warnings for years now
wrote last edited by [email protected]Lol. I feel your pain.
I setup a 2.5TB RAID box in 2011, thought it was going to last a while.
Now my server has a single 8TB data drive, my NAS is 7TB, and I have 2 4TB drives and everything is replicated between them.
Now I need to build another NAS as all this stuff is aging.
-
Background: I've been writing a new media server like Jellyfin or Plex, and I'm thinking about releasing it as an OSS project. It's working really well for me already, so I've started polishing up the install process, writing getting started docs, stuff like that.
I'm interested in how other folks have set up their media libraries. Especially the technical details around how files are encoded and organized.
My media library currently has about 1,100 movies and just shy of 200 TV shows. I've encoded everything as high quality AV1 video with Opus audio, in a WebM container. Subtitles and chapters are in a separate WebVTT file alongside the video. The whole thing is currently about 9TB. With few exceptions, I sourced everything directly from Blu-ray or DVD using MakeMKV. It's organized pretty close to how Jellyfin wants it.
What about you?
2.71Tb/515 series for TV, 6.28Tb/1176 titles in Movies.
Almost everything in MKV because that's what I prefer.
I use Plex so it's organized according to their requirements.
Everything is stored with a redundant backup on a Synology NAS with 6/9 HDD bays filled, totaling 48Tb in total storage space.
I run two servers (one on the Synology, one on a NUC-type Asus box) along with all my other systems.
Oh, and I have dual antenna tuners connected as well for live TV, DVR and playback.
-
Nice write-up. I thought I had a large library (24TB) and my off site backup is starting to get full. I backup everything though but I have long debated on if there's a point of keeping movies and TV since they'll likely always be available. Anyway, I never thought of generating a list of files and eliminating the stuff that's not particularly important. Good idea.
I used to back everything up before I broke the 50TB mark. Just can't justify it now. I even looked at LTO drives for backing up the multimedia but they're still to expensive for the higher capacity drives. And then you need tapes..
All the truely good content will always be out there somewhere on the net.
The script I use to generate the file lists is very very basic. Nothing special no formatting the lists or anything since it's just for that oh balls, everything is gone scenario.
ls -alR /mnt/volume1/media > /mnt/volume2/backups/file_lists/media.txt
ls -alR /mnt/snapraid/data* > /mnt/volume2/backups/file_lists/snapraid.txt
Those text files are also part of the files backed up with PBS so I can always go back and restore previous versions of them. You may ask why I generate the list twice? The first is just everything inside the media folder on the volume1 mount point. The second let's me see what files are on each individual drive so if only 1 drive dies I can just grep the text file and output to another text file and show me what is on that 1 drive.
-
Background: I've been writing a new media server like Jellyfin or Plex, and I'm thinking about releasing it as an OSS project. It's working really well for me already, so I've started polishing up the install process, writing getting started docs, stuff like that.
I'm interested in how other folks have set up their media libraries. Especially the technical details around how files are encoded and organized.
My media library currently has about 1,100 movies and just shy of 200 TV shows. I've encoded everything as high quality AV1 video with Opus audio, in a WebM container. Subtitles and chapters are in a separate WebVTT file alongside the video. The whole thing is currently about 9TB. With few exceptions, I sourced everything directly from Blu-ray or DVD using MakeMKV. It's organized pretty close to how Jellyfin wants it.
What about you?
650 shows, 1400 movies, 1450 anime. Take up like 130TB or something
-
Why start anew instead of forking or contributing to Jellyfin?
The short answer is because it's a fun project, and I wanted to see if I had it in me to make exactly the media server I want.
The longer answer is that I wanted something dramatically and fundamentally different from what either Jellyfin or Plex have to offer.
- Can run without breaking a sweat on junk/old/cheap hardware like a Raspberry Pi or old laptop.
- Can be safely Internet-facing -- no anonymous access, and no web-based admin features or API.
- Hyper-lean and minimal. All-in, I wanted something on the order of 1MB for client app, server, all dependencies, everything.
I don't see either of those goals happening with a contribution or fork, because achieving them would require some dramatic feature deprecation.
-
The short answer is because it's a fun project, and I wanted to see if I had it in me to make exactly the media server I want.
The longer answer is that I wanted something dramatically and fundamentally different from what either Jellyfin or Plex have to offer.
- Can run without breaking a sweat on junk/old/cheap hardware like a Raspberry Pi or old laptop.
- Can be safely Internet-facing -- no anonymous access, and no web-based admin features or API.
- Hyper-lean and minimal. All-in, I wanted something on the order of 1MB for client app, server, all dependencies, everything.
I don't see either of those goals happening with a contribution or fork, because achieving them would require some dramatic feature deprecation.
All-in, I wanted something on the order of 1MB for client app, server, all dependencies, everything.
Okay that's gotta be radically different!
