@mitch @dave parsing seems stable by querying for updated feeds every minute. Average seems more like ~500 updated feeds per minute.

I’m surprised there are that many new episodes released online every minute. I thought I saw ~500 updated feeds every 10 minutes previously. I wonder if I messed up or something changed from the time I made that estimate?

Anyway, we seem stable now, but I definitely need to make our recent feeds firehose implementation more robust...

Show thread

@dave it makes me wonder if anything has changed drastically recently? A week ago every 10 minutes was returning ~500 updated feeds for me. Now every 1 minute is returning ~700 updated feeds.

Show thread

@dave uh oh, actually even if I change the interval to 1 minute since time, I am getting ~700 updated feeds in that time. I can't use 2 minutes since ~400 feeds will be missing. I'm going to change the interval to 1 minute, but it appears even that is too long to be reliable.

Show thread

@mitch @dave I’m going to change it to 5 minutes since it’s missing feeds right now, but lmk if you have thoughts/concerns

Show thread

@dave I've been asking the recently updated feeds endpoint for the max of 1000 feeds updated in the last 10 minutes, but it appears that time scale is not short enough to ensure less than 1000 feed results, so we're missing updates.

Can I reduce the interval that we are pinging that endpoint? How small can I make it without spamming?

@dave did I hear there is a podcast rankings endpoint? For sorting by popularity?

Database question:

I wanted to load our prod database on an external hard drive for local debugging purposes, and I did, but the process took over 24 hours to complete.

Basically I created the sql backup using pg_dump, then ran

psql -f /path/to/backup.sql

Does anyone know a faster way to restore a database from an sql file?

Since syncing our db with PI, search queries take way too long. They finish quick the second time you make a query (apparently bc the table is sorted in memory then) but the first request can take 20+ seconds.

My friend said our problem is the search query has a wildcard in the front, so the database can't use an index on the title column. The command:

'LOWER(podcast.title) LIKE :title':

title = '%Podcasting 2.0%'

I know it's a deep subject but anyone have advice for making search better?

It just dawned on me listening to Podcasting 2.0: Episode 10 that if the money flows, what would be to stop music artists from releasing chapters (tracks) of episodes (albums) and taking control of their own livelyhoods with friendfollowing to give them an audience.

Where would that take the world?
adam.curry.com/html/PC20102020

The first podcast to use the <podcast:soundbite> tag with a title has been added to Podverse 🎉

We now have over 90000 soundbites in our database thanks to BuzzSprout, but soundbites only get featured on our home page if they have a title. The "Dudes And Dads Podcast" is the first podcast we know of to use the soundbite title attribute.

podverse.fm/clip/64h49iwo4j

An endpoint for retrieving crowdsourced clips is now live on Podverse! Provide the mp3 url for an episode to this endpoint and get a JSON containing all of its crowdsourced clip data.

For a demo check out the annotations for Podcasting 2.0 Episode 12:

api.podverse.fm/api/v1/clips?m

Questions and feedback appreciated!

@hypercatcher @martin @adam @martin

Podverse is now syncing with the Podcast Index 🎉 thanks to PI our database has grown from 90,000 podcasts to over 1.3 million! @adam @dave

@adam

AC,

I have stopped dev of Apps page (everything works except for app-specific icons) and as described in previous toot i leave it up to @dave to integrate it via pull request or via downloads from my fork.

The app is operational in my testbed in AWS:

podcastindex.herokuapp.com/app

Next: once Dave will merge this Apps page into main repo, I can fork it anew and I can help with further dev next weekend.

Also: I'm availble for Dave 24/7 for react/other support via jitsi or here - just ask!

@cisene maybe this is something PodcastIndexId can solve? We could use the id as a universal identifier to provide a single button on podcast websites that either 1) opens that podcast in whatever that person’s default podcast app is, or 2) shows the user how to set a default podcast app on their phone? Is that possible on iOS/Android today?

I registered newpodcastapps.com and am pointing it to the GitHub page until we have a nice page in the index to promote everything.

Hosting companies supporting the namespace also need to be listed.

Let’s get some traffic flowing to y’all!

Does anyone have theories as to what could be causing this? Signal 9? I'm using Docker and Postgres.

Show thread

Dev question (Postgres):

I'm trying to load our fairly large prod Postgres database locally. I'm using an exported .sql file generated by pg_dump. In order to import the file I am using:

psql -W -f ../postgres.sql

However, I've run this 3 times, and after about 20 minutes, I get this crash:

LOG: background worker "parallel worker" (PID 94) was terminated by signal 9: Killed

Failed process was running: ALTER TABLE ONLY public.episodes
ADD CONSTRAINT "PK_blah" PRIMARY KEY (id)

New API Endpoint:

api.podcastindex.org/api/1.0/r

Pulls the most recent 60 soundbites from the index. Might be good for some fun.

Show more
PodcastIndex Social

Intended for all stake holders of podcasting who are interested in improving the eco system