Follow

Can anyone recommend an / that is fast and predictable?

· · Web · 1 · 0 · 0

@UncensoredNews That's what I was using, honestly I found it quite disappointing.

I ended up writing my own in C++ and adding python bindings on top.

@gabmus It is very helpful to know that you were disappointed with #Python's #Feedparser.

Specifically what bug did you find? Is it now listed in the isses section on github?

My software stack is way way too big to be rewriting feedparser.

I hope you open source your c++.

@UncensoredNews Feedparser is slow particularly if you have lots of feeds to parse.

Plus, it doesn't do much parsing, as it doesn't take into account tag variations across different standards to unify them, instead you have to do the heavy lifting. I was looking for something "magical": you give it a feed, and it gives you the relevant metadata it manages to find.

The C++ lib I made is indeed open source, you can find it at gitlab.com/gabmus/syndication- but while it does work, it's still unrefined.

@UncensoredNews It's also very much a possibility that I was just using it wrong, but regardless the speed argument remains, as most of the reason for the incredible speed boost I'm getting in C++ has to do with the fact that Python is, sadly, slow.

In my head it makes sense to do the heavy lifting in a faster language, plus it gave me the chance to learn how to create python bindings for C/C++ code.

@gabmus
Very interesting. Right now I have maybe 100 feeds, maybe it will eventually it will go to 1000 feeds.

An obvious question is what are the other C/c++/Go feed parsers, and how do they compare to your library and #python 's #feedparser

@UncensoredNews couldn't find a suitable existing feed parser for C++, do let me know if you know any.

If I had found something else I wouldn't have made my own after all.

@gabmus
I think I count 88 libraries in my requirements.txt file, so this is a small part of my problems. Moreover, I have all day to be downloading and parsing feeds, so no need for speed.

I am sure there will be a good #golang RSS parser.

I also think that there is opportunity for upgrading #python libraries.

Best of luck with your project.

@UncensoredNews I'm building a feed reader, so speed is quite important, users can't be expected to wait minutes to have a couple hundred feeds to load.

Maybe feedparser is just designed for different use cases.

@gabmus
That makes sense. Then you need speed. I have a preselected set of feeds I recommend to users so that it can take as much time as needed. Just wastes CPU cylces. That is acceptable.
#rss #python

Sign in to participate in the conversation
LinuxRocks.Online

Linux geeks doing what Linux geeks do...