If anyone has some gigantic podcast feeds they know of send them my way (podcast index id’s or urls). I am stress testing the initial design of the new aggregator for speed and async sanity.
@dave@podcastindex.social probably behind the schemes is a good choice
@cbrooklyn112 Already got it.
Thanks for all the feeds. Initial work shows 3 seconds to parse 200 megabytes worth of feeds. That’s just handling a few initial node types.l but it’s encouraging since the files are still coming from disk, so there is that overhead as well that will be reduced.
@dave when reading the files are they compresses?
Either by the Filesystem or something like gzip/zstd
@suorcd Nope.
@dave how hard would it be to try?
I think you'd be able to get faster results, with very little overhead.
If not and your work is ready for another tester, I have the hardware.
@suorcd I can post the code if you'd like to test.
@dave please do.
Repo for the new parser code sandbox. If you already have Rust, just clone the repo and `cargo run`.
@dave This one is "fun". 1184 items, with the final line 80K characters long.
https://podcastindex.org/podcast/311515
@dave This oneis massive. And good, but thatis probably not relevant to your needs.
@dave a few more...
793331
1227360
743229
449674
56175
@dave 2589 items for https://podcastindex.org/podcast/315983 by @emilcar
PI Shows 2500
@dave You could try "The Kyle & Jackie O Show". It has over 6,200 episodes in it's RSS feed.
@dave NY times daily, over 2k episodes https://feeds.simplecast.com/54nAGcIl
@aegrumet Perfect.