Commit Graph

50 Commits

Author SHA1 Message Date
Eric Le Lay 831186b0c7 fix overlooked stream.text in conversion to util.response_text 2020-07-23 22:33:15 +02:00
Eric Le Lay 12912bed3e fix naive access to response.text
bytes for xml, utf-8 fallback for html, known encoding if available
2020-07-18 15:04:04 +02:00
Eric Le Lay d59bba5b6d fix invalid encoding used to parse feeds
eg. http://podcast.hr-online.de/hrinfo_wissenswert/podcast.xml
2020-07-14 18:45:38 +02:00
Eric Le Lay 48d851d292 rework feedcore.Fetcher interface to allow reusing it in custom channel parsers 2020-07-14 18:43:44 +02:00
Eric Le Lay a56651ad79 fix linter warning 2020-07-12 17:06:34 +02:00
Eric Le Lay 3babb86c70 use requests package for util.urlopen
now a requests.Response is returned instead of the file-like object from urllib.
Fixed all usages of util.urlopen: it simplifies getting json, text encoding detection.
In particular feedcore (responsible for fetching feeds) is simplified.

This is a first pass and could benefit from better usage of the requests api
(Sessions for instance, to keep connection pools)

TODO: download.py
2020-07-11 17:42:28 +02:00
auouymous 15505a3d05 fix incorrectly sorted imports 2020-02-05 00:40:37 -07:00
Eric Le Lay dfb245fd85 don't override custom feed descriptions 2019-08-27 21:55:49 +02:00
Eric Le Lay 034e3ffa2b Merge default and custom feed handling; support paged feeds
RFC 5005 (http://podlove.org/paged-feeds/)

Fixes #231
2019-06-09 18:59:09 +02:00
Eric Le Lay f51ef5a88e fix #571 unable to subscribe to authenticated podcast after redirect 2018-12-15 17:02:16 +01:00
MarkusHackspacher 82f41f4c21 sorting imports
with isort -y -rc
see #393
2018-07-24 11:08:10 +02:00
MarkusHackspacher eecfd64f63 fix E261 at least two spaces before inline comment 2018-05-17 08:39:56 +02:00
MarkusHackspacher 0c18a43d13 fix E302 and E303, only blank line change 2018-02-11 00:22:00 +01:00
MarkusHackspacher 955499fcd1 fix E305 expected 2 blank lines after class or function definition 2018-02-10 11:11:20 +01:00
Thomas Perl 98ec45a712 Update copyright years (2018) 2018-01-28 19:39:53 +01:00
Thomas Perl 7b46a0ee72 Update copyright years (2017) 2017-12-16 13:54:59 +01:00
Eric Le Lay 858497936f Merge branch 'master' into gtk3
- soundcloud non downloadable tracks
- try to parse feeds even if content-type says HTML
2017-11-27 22:27:41 +01:00
Mel Collins 8fb2c35e64 Combine feed autodiscovery conditionals 2017-04-02 13:24:36 +02:00
Mel Collins 8e70acaceb Try to parse feeds even if content-type says HTML 2017-04-01 16:22:26 +02:00
Adam Voss d9e6234e6e Merge Fix file:// URI scheme support (Fixes #206) 2017-03-25 21:16:16 -05:00
Adam Voss 7ed1c0fe50 decode bytes of text/html feed to str 2017-03-03 21:56:12 -06:00
Thomas Perl 6fb7c9c757 Fix file:// URI scheme support (Fixes #206) 2017-01-15 13:16:18 +01:00
Thomas Perl 0122e8cc00 Merge branch 'master' into gtk3 2016-12-18 13:12:04 +01:00
Thomas Perl 3068234313 Fix HTTP conditional GET (Fixes #193, #182) 2016-12-18 13:04:25 +01:00
Thomas Perl 5a7c30b359 Port to Python 3 2016-11-21 23:13:46 +01:00
Thomas Perl 4189cf48fb Port from feedparser to podcastparser 2016-11-18 14:47:15 +01:00
Markus Wamser 0d5520a462 fix https://bugs.gpodder.org/show_bug.cgi?id=2023
https://bugs.gpodder.org/show_bug.cgi?id=2023 is very annoying, as podcast stopped updating quietly (as long as the offending entry is in the feed). It was also confusing as adding such a failed.

Example taken from http://www.br-online.de/podcast/kalenderblatt/cast.xml
    <item>
      <title>Kaufhaus Schocken in Stuttgart abgerissen - 02.05.1960</title>
      <pubDate>Mon, 02 May 1960 09:05:01 +0100</pubDate>
      <description>Das Auge kauft mit! Das gilt nicht nur für die Ware, sondern für das Shoppingerlebnis insgesamt. Das denkt sich Salman Schocken und baut seine Kaufhäuser zu architektonischen Meisterwerken aus. Autorin: Leo Hoffmann</description>
      <link>http://cdn-storage.br.de/iLCpbHJGNL9zu6i6NL97bmWH_-by/_-ZS/9-k6528G/600502_0905_radioWissen_Kaufhaus-Schocken-in-Stuttgart-abgerissen.mp3</link>
      <enclosure url="http://cdn-storage.br.de/iLCpbHJGNL9zu6i6NL97bmWH_-by/_-ZS/9-k6528G/600502_0905_radioWissen_Kaufhaus-Schocken-in-Stuttgart-abgerissen.mp3" length="3434832" type="audio/mpeg"/>
      <guid isPermaLink="false">600502_0905_radioWissen_Kaufhaus-Schocken-in-Stuttgart-abgerissen.mp3</guid>
      <itunes:author>Bayern 2</itunes:author>
      <itunes:summary>Das Auge kauft mit! Das gilt nicht nur für die Ware, sondern für das Shoppingerlebnis insgesamt. Das denkt sich Salman Schocken und baut seine Kaufhäuser zu architektonischen Meisterwerken aus. Autorin: Leo Hoffmann</itunes:summary>
      <itunes:keywords>Kaufhaus Schocken, Stuttgart, Salman Schocken, Einzelhandel, Shopping, Architektur, Erich Mendelsohn, Wissen, Geschichte, Das Kalenderblatt, radioWissen, Feuilleton, Podcast, Bayern 2, Bayerischer Rundfunk</itunes:keywords>
      <itunes:duration>00:03:34</itunes:duration>
    </item>
2016-04-08 00:21:56 +02:00
Thomas Perl 36eefc8a30 Update copyright years (2016) 2016-01-15 15:22:52 +01:00
Thomas Perl aa72634950 Update copyright years (2015) 2015-05-24 19:33:39 +02:00
Thomas Perl 553e35f219 Update copyright years (2014) 2014-02-27 21:55:13 +01:00
Thomas Perl bda982be02 Update copyright years for 2013 2013-03-05 16:38:48 +01:00
Thomas Perl 5b552131f8 Cleanup feed parsing, remove some special-cases 2012-12-28 15:34:20 +01:00
Thomas Perl c25992445e Fix missing ".feed" in feedcore (bug 1672) 2012-09-23 21:08:24 +02:00
Thomas Perl e5f778ced9 feedcore: Fix feed autodiscovery (bug 1672) 2012-09-23 20:17:47 +02:00
Thomas Perl 4481584db8 Feedcore: Check feedparser version (bug 1648) 2012-08-15 10:56:04 +02:00
Thomas Perl f2f187549d Feedcore: Successful updates should return value 2012-07-10 11:57:59 +02:00
Thomas Perl ee2a5b7e89 Feedcore: Improve parsing of publishing date
This works around problems with invalid date formats
and also prefers the "published" field over "updated".

See also Feedparser issue 327:
http://code.google.com/p/feedparser/issues/detail?id=327
2012-02-12 11:38:26 +01:00
Thomas Perl 5205a4b71d Update copyright years (add 2012) 2012-01-09 21:19:24 +01:00
Thomas Perl 57051e788f Feedcore: Cleanup unused code parts 2012-01-02 15:17:03 +01:00
Thomas Perl 876acad2e1 YouTube: Fix pubDate via monkey-patching (bug 1492)
This is very dirty, but it works for now, and avoids
the annoying sorting/publishing issues with YouTube.
2011-12-05 09:57:49 +01:00
Thomas Perl 9f695aedb3 Support for RSS redirects in feedcore (bug 1292)
Thanks to Ilkka Laukkanen for the initial feature
request and for Kurt McKee and Stefan Kögl for the
implementation in feedparser (since 5.0).

This feature will only work with feedparser >= 5.0,
but will gracefully degrade (ignore RSS redirects)
with older versions of feedparser.
2011-08-08 21:51:19 +02:00
Thomas Perl 4376057001 Update copyright years 2011-04-01 18:59:53 +02:00
Thomas Perl 495b1658f5 Add support for file:// URLs for feeds (bug 1229) 2010-12-16 19:24:06 +01:00
Thomas Perl 53f83892d3 Update website URL and e-mail address 2010-11-22 19:40:29 +01:00
Thomas Perl cf44f7c30e Code cleanup: Remove unused imports 2010-05-17 18:26:13 +02:00
Thomas Perl 181564f1b7 Add 2010 to the years in copyright notice
The following command has been used:

grep -r '2005-2009 Thomas Perl and the gPodder Team' * | cut -d: -f1 |
xargs sed -i -e 's/2005-2009 Thomas Perl and the gPodder Team/2005-2010
Thomas Perl and the gPodder Team/g'
2010-01-02 17:35:42 +01:00
Thomas Perl ef10a33a47 Add EA.com compatibility to feedparser (bug 588)
Modify feedparser's default "Accept:" header to
also include a plain "*/*" content type, so that
the web server at EA.com will happily return the
RSS feed contents instead of an error.

Thanks to Romain Janvier for reporting this bug.
2009-10-05 19:27:25 +02:00
Sam Freilich cb0fa30e23 Fix adding password-protected feeds (bug 544)
Don't require feed.version when the feed could
not be retrieve due to authentication request.
2009-09-05 19:19:36 +02:00
Morten Nygaard Åsnes 75237d2d96 Monkey-patch feedparser to support Media RSS (bug 507)
Support Media RSS' media_content tag for Media RSS podcasts.

See http://code.google.com/p/feedparser/issues/detail?id=100#c4
2009-07-23 22:26:07 +02:00
Thomas Perl 4c7088dfa3 First cut of new feedcore module + related changes
This might break a lot, but it makes feed parsing and
updating so much cleaner, and also helps with error
reporting, because exceptions are bubbled up to the UI.

Removed:

 * Changing feed URL (this introduces too many problems)
 * Support for FTP proxy servers (who uses FTP for podcasts?)
2009-06-12 00:51:13 +02:00