Commit graph

314 commits

Author SHA1 Message Date
jesopo
b7dd78ef1a restore 5 second (instead of default 10) deadline for http.request 2019-09-17 13:44:14 +01:00
jesopo
94c3ff962b use utils.deadline_process() in utils.http._request() so background threads can
call _request()
2019-09-17 13:41:11 +01:00
jesopo
fa95eaa9eb add .get() to CaseInsensitiveDict 2019-09-17 13:40:37 +01:00
jesopo
d454f9b732 use Queue.get() with timeout, not Process.join() for timeout
this was because the threads spawned by multiprocessing.Queue seemed to be
making Process.join() believe the subprocess had not exited.
2019-09-17 13:39:23 +01:00
jesopo
1ed14f9a17 first draft of multiprocess.Process deadline system 2019-09-17 11:56:30 +01:00
jesopo
334d580c57 'seperate_hostmask()' -> 'parse_hostmask()' 2019-09-16 18:43:57 +01:00
jesopo
47735421b8 add json_body arg to Request to json-encode body, only return from body if
not null
2019-09-16 10:57:18 +01:00
jesopo
f9d13dc373 support '0' as an IntSetting value 2019-09-15 22:22:30 +01:00
jesopo
d950eb3660 add utils.SensitiveSetting, to .format() hide value 2019-09-12 12:17:31 +01:00
jesopo
ba0911f2e7 add utils.Setting.format() so subtypes can format differently 2019-09-12 12:17:09 +01:00
jesopo
9d33354d16 translate INVITE from [channel_name, target] to [target, channel_name] 2019-09-12 11:24:25 +01:00
jesopo
540c7b8c44 Revert "INVITE should be [channel_name, target]"
This reverts commit f3d8ffad2c.
2019-09-12 11:23:29 +01:00
jesopo
f3d8ffad2c INVITE should be [channel_name, target] 2019-09-12 11:21:29 +01:00
jesopo
77f50187c5 allow Requests to specify a useragent 2019-09-12 10:41:50 +01:00
jesopo
9d6a3982ed add a helper utils.http.Client static object 2019-09-11 17:53:49 +01:00
jesopo
51dc26d113 add proxy to Request objects 2019-09-11 17:53:37 +01:00
jesopo
4a97c9eb0d refactor utils.http.requests to support a Request object 2019-09-11 17:44:07 +01:00
jesopo
8f8cf92ae2 automatically decode certain http content types 2019-09-11 15:28:13 +01:00
jesopo
a9b106c6be Don't try to .decode non-html things, default iso-lat-1 for non-html too 2019-09-09 16:17:26 +01:00
jesopo
b83f5d9e30 add flag to disable encoding detection 2019-09-09 14:59:08 +01:00
jesopo
5ef2b7af27 'str.split' -> 's.split' 2019-09-09 14:53:11 +01:00
jesopo
1df82c1cb2 still default to iso-latin-1 if no on-page or in-header content-type is present 2019-09-09 14:48:26 +01:00
jesopo
0a67659637 only look for <meta>-related tags when there are meta tags 2019-09-09 14:39:19 +01:00
jesopo
0a1077c5cd add explicit None return for _find_encoding (mypy) 2019-09-09 14:25:01 +01:00
jesopo
ff9c82bf67 change utils.http.request to best-effort detect on-page encoding
closes #113
2019-09-09 14:11:18 +01:00
jesopo
007bb78d30 make utils.from_pretty_time() format much stricter 2019-09-04 11:22:56 +01:00
jesopo
397cfa8e7e correctly qualify DeadlineExceededException namespace 2019-09-03 14:54:59 +01:00
jesopo
b7b2f31c1c use utils.deadline() in utils.http.request, not raw sigalrm 2019-09-02 15:50:21 +01:00
jesopo
d42d694e64 move deadline alarm time check inside try/finally 2019-09-02 15:50:12 +01:00
jesopo
9cc1ee98eb Pass the content of a webpage to HTTPParsingException 2019-09-02 13:27:44 +01:00
jesopo
408b89aeb7 use \S+ for url regex (for non-ascii chars), use url_sanitize to catch <> 2019-09-02 13:25:48 +01:00
jesopo
9e43ea1599 add per-user fediverse setting 2019-09-02 08:51:56 +01:00
jesopo
722d6418c8 switch to using signal.setitimer so we can restore previous timer 2019-08-30 18:36:48 +01:00
jesopo
a57a06b1cc switch utils.deadline to a context manager 2019-08-30 18:13:12 +01:00
jesopo
e97e3c8f30 add utils.deadline(func, seconds) to put hard limits on code exec time 2019-08-30 17:17:03 +01:00
jesopo
4bfb4c3200 Add IntRangeSetting, reorder OptionsSetting arg order 2019-08-30 14:40:54 +01:00
jesopo
58a4fd7e74 round microseconds up to a whole number and zero-pad to 3 chars 2019-08-30 13:46:00 +01:00
jesopo
d3ed08e336 Don't use 'Z' for "utc timezone" - actually parse timezone out of datetimes 2019-08-30 13:25:19 +01:00
jesopo
b6361ba4e3 Remove extraneous space 2019-08-07 14:00:56 +01:00
jesopo
20042edfd9 Allow bypass of content-type check in utils.http.request 2019-08-05 15:41:02 +01:00
jesopo
d093027431 not all HTTP responses have content-type 2019-08-02 17:33:16 +01:00
jesopo
77dfc76591 switch to function/module magic being a single object 2019-07-26 11:58:06 +01:00
jesopo
c19c6c0e14 asyncio.gather -> asyncio.wait (with timeout) 2019-07-08 14:50:11 +01:00
jesopo
469c725675 tell asyncio.gather which loop to use 2019-07-08 14:41:12 +01:00
jesopo
a1438abf66 close event loop when we're done with it (request_many()) 2019-07-08 13:59:48 +01:00
jesopo
81c7af8ab5 Don't try/except async http exceptions 2019-07-08 13:51:02 +01:00
jesopo
ee0ec0eca1 switch request_many() to use asyncio.gather 2019-07-08 13:46:27 +01:00
jesopo
b62ba469d7 catch async exceptions in utils.http.request_many() 2019-07-08 13:18:59 +01:00
jesopo
078681eddf add missing schema in utils.http.sanitise_url, use in rss.py 2019-07-08 12:54:06 +01:00
jesopo
ecb8364d0d switch to using asyncio's event loop 2019-07-08 12:45:10 +01:00
jesopo
15e143fcff implement utils.http.request_many as a tonado ioloop yield 2019-07-08 11:43:09 +01:00
jesopo
637067c62c url_validate() -> url_sanitise() 2019-07-02 14:15:49 +01:00
jesopo
534854127b Add utils.http.url_validate() for best-effort url tidying 2019-07-02 14:10:18 +01:00
jan6
b6e9713a14 fixed capabilities 2019-07-01 14:41:47 +03:00
jesopo
19f716ab4c Implemnt _options_factory for OptionsSetting to have dynamic options 2019-06-29 22:26:28 +01:00
jesopo
b4c762eb4e Automatically format example text for OptionsSetting (showing options) 2019-06-29 21:33:26 +01:00
jesopo
7091860e54 Setting.__init__ call in OptionsSetting ctor needs self param 2019-06-28 23:27:56 +01:00
jesopo
756396d758 Implement utils.OptionsSetting 2019-06-28 23:26:42 +01:00
jesopo
ae9d099a41 Refactor set/channelset/serverset/botset in to 'utils.Setting' objects 2019-06-28 23:16:05 +01:00
jesopo
f9eb017466 message arg for HTTPWrongContentTypeException/HTTPParsingException 2019-06-28 23:01:21 +01:00
jesopo
97810db8df Give descriptions to utils.http.HTTPException subclasses 2019-06-27 18:28:08 +01:00
jesopo
3c3f0449b6 add CaseInsensitiveDict.__contains__ 2019-06-26 17:57:49 +01:00
jesopo
16d331dd43 add allow_redirects kwarg to utils.http.request() 2019-06-26 17:53:16 +01:00
jesopo
c5785a2d14 implement @utils.kwarg() magic, use it for command.regex hooks 2019-06-26 14:37:41 +01:00
jesopo
90c90e5bbd Implement dependency system for CAPs 2019-06-21 18:05:11 +01:00
jesopo
8a0d99f969 Adda system of aliases for CAPs, mostly for changeable draft specs and creating
dependence between moving specs
2019-06-21 17:15:46 +01:00
jesopo
01ddc04bb9 fix hostmask_match_many type hinting 2019-06-20 17:58:49 +01:00
jesopo
b6e194a450 Implement hostmask_match_any, for more efficient matching of multiple hostmasks 2019-06-20 14:21:11 +01:00
jesopo
792a42be6e Implement utils.irc.hostmask_match() as regex 2019-06-20 14:10:10 +01:00
jesopo
9457430c15 Add MessageTag.present, to see if a tag is present (even without value) 2019-06-19 22:36:42 +01:00
jesopo
ced59a0f74 Allow channel-access and channel-mode check for channels people are not in 2019-06-16 20:48:31 +01:00
jesopo
919a488e86 Implement client-to-server BATCHes 2019-06-16 09:52:54 +01:00
jesopo
8ab4880c5c Change from yield checks to a func in events that pass up EventErrors 2019-06-15 18:42:14 +01:00
jesopo
e0d99fe8b1 Fix utils.Check args type hinting 2019-06-14 12:12:38 +01:00
jesopo
f05fc209b0 Add a way to __or__ utils.Checks, as a "if one of these passes" thing 2019-06-14 12:01:55 +01:00
jesopo
d7fa2cfa24 Catch yields in command callbacks for e.g. permission checks 2019-06-14 11:42:12 +01:00
jesopo
6c5dc958d0 Add .assure() to ParsedLine, to make it immune from .valid() 2019-06-10 10:45:02 +01:00
jesopo
2d46fe0cbf labels: Use BatchType to match, pass lines not line, batch ID isn't a label,
fire event on label response
2019-06-04 17:54:19 +01:00
jesopo
2470c1ec03 Add new BatchType object, to match like how Capability and MessageTag do 2019-06-04 17:51:55 +01:00
jesopo
a802e66dcf Defer decoding http payload bytestring until after checking ContentType 2019-06-04 13:47:03 +01:00
jesopo
ef221f4165 v1.8.0 release 2019-06-03 13:30:39 +01:00
jesopo
c212d70b68 Hold on to BATCH args, allow event hooks to manipulate batches 2019-06-02 15:18:36 +01:00
jesopo
5204ac3300 fix some CAP related type hints 2019-06-02 10:14:09 +01:00
jesopo
f9f637e084 Remove IRCSendBatch, fix some batch-related type hints 2019-06-02 10:13:51 +01:00
jesopo
367eb1c60e Make labeled-resposnes it's own module, tag and track every sent line 2019-05-30 15:36:10 +01:00
jesopo
0be9046669 Pass str object to BeautifulSoup, not bytes. closes #56 2019-05-28 10:22:35 +01:00
jesopo
eeee2458d3 Fix utils.parse.hashflags type hints 2019-05-25 21:42:42 +01:00
jesopo
d0634bb54e Don't store hashflags as a dict before returning them - allow duplicate keys 2019-05-25 12:58:07 +01:00
jesopo
ab1074cf4d Remove mention of ipv4 - detect address family automatically 2019-05-21 10:11:09 +01:00
jesopo
db33e12c8c Remove Capability.enabled() - it's unused 2019-05-20 16:36:02 +01:00
jesopo
b12c38c326 typing.List -> typing.Sequence 2019-05-20 16:35:50 +01:00
jesopo
e752b60f6c use fnmatchcase( so the strings are not case-"normalised" 2019-05-19 16:28:46 +01:00
jesopo
c607a0e67c Implement utils.irc.hostmask_match for glob-like hostmask matching 2019-05-19 16:24:38 +01:00
jesopo
58d6b16744 Remove _name and _draft_name (unused) and add Capability.copy() 2019-05-19 11:00:58 +01:00
jesopo
eccd43d357 Add a way to not put spaces between docstring lines when stripping \n 2019-05-18 18:34:27 +01:00
jesopo
38ba656bda Add type hints to utils.irc.MessageTag 2019-05-14 23:07:42 +01:00
jesopo
421d9c10c7 Add utils.irc.MessageTag and use it for both draft and ratified msgid 2019-05-14 13:02:45 +01:00
jesopo
d291cd5063 Revamp how CAPs are tracked through REQ and ACK/NAK etc 2019-05-11 18:22:40 +01:00
jesopo
295f7a751c Refactor CAPs in to objects 2019-05-11 15:11:09 +01:00
Patrick Nappa
2c344c9ddd forgot the beautiful % 2019-05-03 13:50:51 +10:00