Commit graph

37 commits

Author SHA1 Message Date
jesopo
e239d7a09c throw helpful errors when RSS pages cannot be decoded 2020-01-31 11:47:00 +00:00
jesopo
2d21dfa229 utils.http.Response.data should always be bytes - add .decode and .soup 2019-11-26 13:42:01 +00:00
jesopo
9abc668c0a don't let RSS shortening failure CRITICAL the whole bot 2019-11-14 11:54:10 +00:00
jesopo
0331b763ff refactor multi-line-to-line normalisation to utils.parse.line_normalise(), use it in rss.py
closes #174
2019-10-10 10:33:18 +01:00
jesopo
cf1002a2a9 strip HTML from RSS entry titles
closes #174
2019-10-10 10:20:44 +01:00
jesopo
7a5edccb98 transient timers shouldn't use the event system 2019-10-08 13:49:55 +01:00
jesopo
957a222f96 actually pass Request objects to utils.http.request_many() from rss.py 2019-09-19 15:04:50 +01:00
jesopo
88a69aaa66 give Requests, use them in utils.http.request_many() 2019-09-19 14:54:44 +01:00
jesopo
d9426abec1 suggest similar feeds when trying to remove an unknown feed
closes #100
2019-08-12 16:07:07 +01:00
jesopo
1db3929de3 check hashed and unhashed IDs, for backwards compatibility 2019-08-12 15:16:46 +01:00
jesopo
ff8b81a761 Store more "seen ids", hash seen ids so they take up a constant space 2019-08-12 15:08:36 +01:00
jesopo
1478b3bb32 !rss read will assume url if there's only url registered for the channel 2019-08-12 12:37:47 +01:00
jesopo
cf76cd4991 typo, 'if seen_ids' -> 'if entries' 2019-08-09 23:23:10 +01:00
jesopo
fc765228eb implement !rss read <url>
closes #94
2019-08-09 16:48:41 +01:00
jesopo
c765a29d09 Don't try to get rss urls when there are none 2019-07-16 10:45:29 +01:00
jesopo
ad7f9a8024 add setting to shorten rss urls (rss-shorten) 2019-07-15 17:46:11 +01:00
jesopo
92511122ff add depends-on hashflags to rss.py 2019-07-15 17:45:48 +01:00
jesopo
33e462d27b use entry["link"] for unique checking if entry["id"] isn't available 2019-07-08 15:56:19 +01:00
jesopo
c6c60a5192 TRACE log RSS timing 2019-07-08 13:46:12 +01:00
jesopo
c60edabd74 Handle request_many() not returning a Response for a url 2019-07-08 13:25:25 +01:00
jesopo
078681eddf add missing schema in utils.http.sanitise_url, use in rss.py 2019-07-08 12:54:06 +01:00
jesopo
1def848233 switch rss.py to use utils.http.request_many 2019-07-08 12:45:30 +01:00
jesopo
cd0a860f5f support rss/atom feeds that have no feed title 2019-07-07 10:28:20 +01:00
jesopo
6014a2b0da WARN log when a RSS feed throws in _check_url 2019-07-04 11:21:53 +01:00
jesopo
130b5d1927 add botset IntSetting for RSS interval 2019-07-04 11:15:01 +01:00
jesopo
4d096aa79d delete rss-hooks setting when it's empty 2019-07-01 21:16:04 +01:00
jesopo
08c723f9d0 We should be parsing out (server, channel) for rss hooks 2019-07-01 21:15:06 +01:00
jesopo
5ea0d9751b Only abort individual RSS feeds when they fail 2019-06-27 18:29:17 +01:00
jesopo
9c5c8cf563 seems i completely forgot try/except syntax 2019-06-27 18:21:07 +01:00
jesopo
95537378b4 Only show Exception message when rss feeds fail 2019-06-27 18:18:39 +01:00
jesopo
14b7a8ae11 exc_info need to be in an except block 2019-06-27 16:00:14 +01:00
jesopo
e30ae803b0 get rss feed xml ourselves so feedparser.parse can't hang on us 2019-06-27 11:56:50 +01:00
jesopo
1defa04ad2 rss-seen-ids should be set per-url or they'll overwrite each other 2019-06-24 20:34:22 +01:00
jesopo
a96bffc79d '_title' -> '_name' 2019-06-24 20:23:36 +01:00
jesopo
8adf320e65 Get RSS feed and mark all IDs as "seen" when first adding the URL 2019-06-24 06:48:48 +01:00
jesopo
e88f23e413 iter through rss entries backwards 2019-06-23 16:22:45 +01:00
jesopo
54796470b1 First version of rss.py 2019-06-23 16:03:15 +01:00