Commit graph

168 commits

Author SHA1 Message Date
jesopo
c655668bbe Add fallback_encoding to utils.http.get_url, in case a page has no implicit
encoding
2018-10-10 23:49:59 +01:00
jesopo
f286f3bf48 .decode data prior to json.loads in utils.http.get_url 2018-10-10 15:25:08 +01:00
jesopo
951c315cec Fix syntax error for throwing a timeout when signal.alarm fires 2018-10-10 15:07:04 +01:00
jesopo
015fa8ddff .decode plaintext returns from utils.http.get_url 2018-10-10 15:06:30 +01:00
jesopo
5b9ffe013d Use signal.alarm to Deadline utils.http.get_url and throw useful exceptions 2018-10-10 14:25:44 +01:00
jesopo
be75f72356 Set a max size of 100mb for utils.http.get_url 2018-10-10 14:05:15 +01:00
jesopo
68f5626189 Change utils.http to use requests 2018-10-10 13:41:58 +01:00
jesopo
c28a41ad21 Remove debug print in src.utils.http 2018-10-09 22:39:34 +01:00
jesopo
f69a1ce7c1 Return response code from utils.http.get_url when code=True and soup=True 2018-10-09 22:16:04 +01:00
jesopo
383767c7fb Support post_data in utils.http.get_url 2018-10-08 12:43:31 +01:00
jesopo
681866339c Support multiple types of comments in utils.get_hashflags 2018-10-07 11:24:13 +01:00
jesopo
096fd05cee Typo in utils, 'line.replace[3:]' -> 'line[3:]' 2018-10-04 16:01:34 +01:00
jesopo
f2d9ff9d3b Checking for the presence of the seperator in the return from .partition is a
more "correct" way of doing this
2018-10-04 15:27:52 +01:00
jesopo
feed251f82 Non-present hashflag values should be None, not empty string 2018-10-04 15:23:35 +01:00
jesopo
8d3255c46d Simplify hashflag parsing 2018-10-04 15:22:09 +01:00
jesopo
75f25db4cd Move 'top 10' logic to utils, add !wordiest to modules/words.py 2018-10-04 05:33:15 +01:00
jesopo
6c1ac52fde Use docstring key, not value, as a key. 2018-10-03 16:00:53 +01:00
jesopo
69d58eede2 Move src/Utils.py in to src/utils/, splitting functionality out in to modules of
related functionality
2018-10-03 13:22:37 +01:00