jesopo
|
16d331dd43
|
add allow_redirects kwarg to utils.http.request()
|
2019-06-26 17:53:16 +01:00 |
|
jesopo
|
a802e66dcf
|
Defer decoding http payload bytestring until after checking ContentType
|
2019-06-04 13:47:03 +01:00 |
|
jesopo
|
0be9046669
|
Pass str object to BeautifulSoup, not bytes. closes #56
|
2019-05-28 10:22:35 +01:00 |
|
Patrick Nappa
|
2c344c9ddd
|
forgot the beautiful %
|
2019-05-03 13:50:51 +10:00 |
|
Patrick Nappa
|
471c11e229
|
ensure that non-url characters not separated by whitespace aren't consumed
|
2019-05-03 13:43:08 +10:00 |
|
jesopo
|
bdcb4b5db2
|
Add missing ":"
|
2019-04-25 17:50:41 +01:00 |
|
jesopo
|
1240b154cb
|
Support interfaces that don't have AF_INET and/or AF_INET6
|
2019-04-25 17:48:51 +01:00 |
|
jesopo
|
7643a962bd
|
Refuse to get the title for any url that points locall
|
2019-04-25 15:58:58 +01:00 |
|
jesopo
|
dffee4d223
|
Move REGEX_URL out of isgd.py and title.py in to utils.http
|
2019-04-24 15:46:54 +01:00 |
|
jesopo
|
197ae2e053
|
Raise a specific exception in utils.http.request for "wrong content type"
|
2019-02-28 23:28:45 +00:00 |
|
jesopo
|
846b881e52
|
Throw ValueError when utils.http.request tries to soup non-html/xml data
|
2019-02-27 15:16:08 +00:00 |
|
jesopo
|
cfaf6864fc
|
Don't try to parse non-html/xml stuff with BeautifulSoup
|
2019-02-26 11:18:50 +00:00 |
|
jesopo
|
2d3bb2b5e8
|
Typo in utils.http.request, 'response_heders' -> 'response_headers'
|
2018-12-11 22:31:14 +00:00 |
|
jesopo
|
5b59740043
|
Pass a dict to utils.CaseInsensitiveDict, not a MutableMapping
|
2018-12-11 22:30:57 +00:00 |
|
jesopo
|
d373edfaae
|
Add missing utils import in utils.http
|
2018-12-11 22:30:05 +00:00 |
|
jesopo
|
793d234a0b
|
'utils.http.get_url' -> 'utils.http.request', return a Response object from
utils.http.request
|
2018-12-11 22:26:38 +00:00 |
|
jesopo
|
b543e31cd2
|
Fix/refactor issues brought up by type hint linting
|
2018-10-30 17:49:35 +00:00 |
|
jesopo
|
e07553c362
|
Add type/return hints throughout src/ and, in doing so, fix some cyclical
references.
|
2018-10-30 14:58:48 +00:00 |
|
jesopo
|
d3231e3282
|
signal.signal timer callback takes 2 args
|
2018-10-25 14:09:19 +01:00 |
|
jesopo
|
c655668bbe
|
Add fallback_encoding to utils.http.get_url, in case a page has no implicit
encoding
|
2018-10-10 23:49:59 +01:00 |
|
jesopo
|
f286f3bf48
|
.decode data prior to json.loads in utils.http.get_url
|
2018-10-10 15:25:08 +01:00 |
|
jesopo
|
951c315cec
|
Fix syntax error for throwing a timeout when signal.alarm fires
|
2018-10-10 15:07:04 +01:00 |
|
jesopo
|
015fa8ddff
|
.decode plaintext returns from utils.http.get_url
|
2018-10-10 15:06:30 +01:00 |
|
jesopo
|
5b9ffe013d
|
Use signal.alarm to Deadline utils.http.get_url and throw useful exceptions
|
2018-10-10 14:25:44 +01:00 |
|
jesopo
|
be75f72356
|
Set a max size of 100mb for utils.http.get_url
|
2018-10-10 14:05:15 +01:00 |
|
jesopo
|
68f5626189
|
Change utils.http to use requests
|
2018-10-10 13:41:58 +01:00 |
|
jesopo
|
c28a41ad21
|
Remove debug print in src.utils.http
|
2018-10-09 22:39:34 +01:00 |
|
jesopo
|
f69a1ce7c1
|
Return response code from utils.http.get_url when code=True and soup=True
|
2018-10-09 22:16:04 +01:00 |
|
jesopo
|
383767c7fb
|
Support post_data in utils.http.get_url
|
2018-10-08 12:43:31 +01:00 |
|
jesopo
|
69d58eede2
|
Move src/Utils.py in to src/utils/, splitting functionality out in to modules of
related functionality
|
2018-10-03 13:22:37 +01:00 |
|