Commit graph

65 commits

Author SHA1 Message Date
jesopo
793d234a0b 'utils.http.get_url' -> 'utils.http.request', return a Response object from
utils.http.request
2018-12-11 22:26:38 +00:00
jesopo
b543e31cd2 Fix/refactor issues brought up by type hint linting 2018-10-30 17:49:35 +00:00
jesopo
e07553c362 Add type/return hints throughout src/ and, in doing so, fix some cyclical
references.
2018-10-30 14:58:48 +00:00
jesopo
d3231e3282 signal.signal timer callback takes 2 args 2018-10-25 14:09:19 +01:00
jesopo
c655668bbe Add fallback_encoding to utils.http.get_url, in case a page has no implicit
encoding
2018-10-10 23:49:59 +01:00
jesopo
f286f3bf48 .decode data prior to json.loads in utils.http.get_url 2018-10-10 15:25:08 +01:00
jesopo
951c315cec Fix syntax error for throwing a timeout when signal.alarm fires 2018-10-10 15:07:04 +01:00
jesopo
015fa8ddff .decode plaintext returns from utils.http.get_url 2018-10-10 15:06:30 +01:00
jesopo
5b9ffe013d Use signal.alarm to Deadline utils.http.get_url and throw useful exceptions 2018-10-10 14:25:44 +01:00
jesopo
be75f72356 Set a max size of 100mb for utils.http.get_url 2018-10-10 14:05:15 +01:00
jesopo
68f5626189 Change utils.http to use requests 2018-10-10 13:41:58 +01:00
jesopo
c28a41ad21 Remove debug print in src.utils.http 2018-10-09 22:39:34 +01:00
jesopo
f69a1ce7c1 Return response code from utils.http.get_url when code=True and soup=True 2018-10-09 22:16:04 +01:00
jesopo
383767c7fb Support post_data in utils.http.get_url 2018-10-08 12:43:31 +01:00
jesopo
69d58eede2 Move src/Utils.py in to src/utils/, splitting functionality out in to modules of
related functionality
2018-10-03 13:22:37 +01:00