I have much more powerful hardware than you but Jellyfin shouldn’t take more than a few seconds to load directories. The hardware is fairly irrelevant, this was still the case when I was on my old nas (which was an ancient pc that was garbage). Jellyfin doesn’t require much. My library is gigantic too, easily over 100,000 items across music, movies, and tv.

What do logs say? How is your network? When I moved into my new place i went ethernet only and had issues with Jellyfin (and other self hosted stuff) and tracked it down to one cable that was super cheap and limited to 100mbit.

Jellyfin isn’t really geared towards viewing media in a folder structure though.

You may be best off going with a freetube access. TBH I find Jellyfin works best with media that can be scraped or that I’m willing to create nfo files for. My music/tv/movies? These are overwhelmingly fine. Every once in a while a niche show or album requires manual scraping or a custom nfo. But I also have some other collections like music videos. Imvdb exists but is far less complete compared to other scraping sources and as a result I don’t even bother using it, the overwhelming majority of my collection needs manual nfos. When I’ve tried to contribute to it my contributions have sat in pending for literal months pending approval, even for obvious videos by major artists.

I don’t know of any scraper for youtube videos and such a thing would be a tremendous undertaking. If you archive a lot it’s a lot of nfos to create. Perhaps you could make a script that generates them automatically by scraping the description and grabbing the thumbnail as fan art?