jesopo
72649a90c2
only BeautifulSoup for finding encoding when it's a html-ish type
2019-09-20 13:38:00 +01:00
jesopo
d609bfb16d
cache calculated "next expiration" time
2019-09-20 11:47:57 +01:00
jesopo
efc0e197e5
Allow passing source Hostmask to IRCBatch
2019-09-19 18:16:10 +01:00
jesopo
e34259f967
log call was replaced with Exception but [] on args remained
2019-09-19 15:30:27 +01:00
jesopo
88a69aaa66
give Requests, use them in utils.http.request_many()
2019-09-19 14:54:44 +01:00
jesopo
d8e3a1c7ee
utils.http.request_() has no self, let alone self.log
2019-09-19 14:02:48 +01:00
jesopo
b69c9146b2
should be using pair_start/pair_end throughout for
2019-09-19 13:51:27 +01:00
jesopo
cd0d39ee5e
also show "bad" data in HTTPParsingException when a message is provided
2019-09-18 14:20:59 +01:00
jesopo
312f8906ae
show "bad" data in HTTPParsingException message
2019-09-18 10:52:05 +01:00
jesopo
a003c97fba
move q.close() to where it will be called even if deadline is hit
2019-09-18 10:24:01 +01:00
jesopo
dce6eee8c9
move _raise_deadline() out of except block to clean up printed stacktrace
2019-09-18 10:21:40 +01:00
jesopo
ee6360be22
don't check already-read data when checking for too-large requests
...
this check was here because the first read will return empty if it was an
invalid byte sequence for e.g. gzip because we needed to receive more data. the
second read will always return data (not decoded) so regardless of what the
already-read data is, the second read is the only criteria we need.
2019-09-17 17:33:23 +01:00
jesopo
1ac7f2697e
log which URL caused an error in request_many
2019-09-17 17:09:19 +01:00
jesopo
98545a9fb4
only decode content-types in DECODE_CONTENT_TYPES
2019-09-17 16:12:03 +01:00
jesopo
8ca0d30fef
Response.__init__() needs encoding now
2019-09-17 14:11:12 +01:00
jesopo
b7dd78ef1a
restore 5 second (instead of default 10) deadline for http.request
2019-09-17 13:44:14 +01:00
jesopo
94c3ff962b
use utils.deadline_process() in utils.http._request() so background threads can
...
call _request()
2019-09-17 13:41:11 +01:00
jesopo
fa95eaa9eb
add .get() to CaseInsensitiveDict
2019-09-17 13:40:37 +01:00
jesopo
d454f9b732
use Queue.get() with timeout, not Process.join() for timeout
...
this was because the threads spawned by multiprocessing.Queue seemed to be
making Process.join() believe the subprocess had not exited.
2019-09-17 13:39:23 +01:00
jesopo
1ed14f9a17
first draft of multiprocess.Process deadline system
2019-09-17 11:56:30 +01:00
jesopo
c23530be89
add missing IRCLine import to IRCChannel
2019-09-17 10:21:31 +01:00
jesopo
334d580c57
'seperate_hostmask()' -> 'parse_hostmask()'
2019-09-16 18:43:57 +01:00
jesopo
03892d0ae3
topic_setter should be stored as a Hostmask
2019-09-16 18:38:46 +01:00
jesopo
a48aececfd
EventRoot._hook's kwarg should be a list of tuples
2019-09-16 15:34:34 +01:00
jesopo
47735421b8
add json_body arg to Request to json-encode body, only return from body if
...
not null
2019-09-16 10:57:18 +01:00
jesopo
f9d13dc373
support '0' as an IntSetting value
2019-09-15 22:22:30 +01:00
jesopo
60ced118a0
get_user_status() -> get_user_modes(), filter in format_activity.py
2019-09-12 22:39:21 +01:00
jesopo
d950eb3660
add utils.SensitiveSetting, to .format() hide value
2019-09-12 12:17:31 +01:00
jesopo
ba0911f2e7
add utils.Setting.format() so subtypes can format differently
2019-09-12 12:17:09 +01:00
jesopo
9d33354d16
translate INVITE from [channel_name, target] to [target, channel_name]
2019-09-12 11:24:25 +01:00
jesopo
540c7b8c44
Revert "INVITE should be [channel_name, target]"
...
This reverts commit f3d8ffad2c
2019-09-12 11:23:29 +01:00
jesopo
f3d8ffad2c
INVITE should be [channel_name, target]
2019-09-12 11:21:29 +01:00
jesopo
77f50187c5
allow Requests to specify a useragent
2019-09-12 10:41:50 +01:00
jesopo
2c5a2d9db3
add IRCChannel.send_invite()
2019-09-12 10:24:02 +01:00
jesopo
2bd7d591c3
Revert "add IRCBot.http_client(), to allow modules to override default http client"
...
This reverts commit 81347fbba0
2019-09-11 18:08:44 +01:00
jesopo
81347fbba0
add IRCBot.http_client(), to allow modules to override default http client
2019-09-11 17:54:24 +01:00
jesopo
9d6a3982ed
add a helper utils.http.Client static object
2019-09-11 17:53:49 +01:00
jesopo
51dc26d113
add proxy to Request objects
2019-09-11 17:53:37 +01:00
jesopo
4a97c9eb0d
refactor utils.http.requests to support a Request object
2019-09-11 17:44:07 +01:00
jesopo
8f8cf92ae2
automatically decode certain http content types
2019-09-11 15:28:13 +01:00
jesopo
8d8d9219b8
don't return "+" when a channel has no modes
2019-09-10 15:10:14 +01:00
jesopo
a9b106c6be
Don't try to .decode non-html things, default iso-lat-1 for non-html too
2019-09-09 16:17:26 +01:00
jesopo
b83f5d9e30
add flag to disable encoding detection
2019-09-09 14:59:08 +01:00
jesopo
5ef2b7af27
'str.split' -> 's.split'
2019-09-09 14:53:11 +01:00
jesopo
1df82c1cb2
still default to iso-latin-1 if no on-page or in-header content-type is present
2019-09-09 14:48:26 +01:00
jesopo
0a67659637
only look for <meta>-related tags when there are meta tags
2019-09-09 14:39:19 +01:00
jesopo
0a1077c5cd
add explicit None return for _find_encoding (mypy)
2019-09-09 14:25:01 +01:00
jesopo
ff9c82bf67
change utils.http.request to best-effort detect on-page encoding
...
closes #113
2019-09-09 14:11:18 +01:00
jesopo
dadefeb8f3
remove try-rejoin (on 477) logic. will move to a module later
2019-09-07 15:21:35 +01:00
jesopo
d973b547e1
add a create flag to IRCChannels.get_id() to optionally not create new ids
2019-09-05 14:13:58 +01:00
jesopo
d7ae182428
cache when a setting isn't set but don't cache "default" value
2019-09-04 15:26:36 +01:00
jesopo
473c2723e7
only cache get_setting value when it's not the default value
2019-09-04 15:24:50 +01:00
jesopo
007bb78d30
make utils.from_pretty_time() format much stricter
2019-09-04 11:22:56 +01:00
jesopo
397cfa8e7e
correctly qualify DeadlineExceededException namespace
2019-09-03 14:54:59 +01:00
jesopo
b7b2f31c1c
use utils.deadline() in utils.http.request, not raw sigalrm
2019-09-02 15:50:21 +01:00
jesopo
d42d694e64
move deadline alarm time check inside try/finally
2019-09-02 15:50:12 +01:00
jesopo
a160148c2f
actually prune non-loadable modules before doing dependency checks
2019-09-02 15:46:31 +01:00
jesopo
43a7b7d949
add basic caching logic for channel settings
2019-09-02 14:07:39 +01:00
jesopo
7959f1ab88
change Cache.py to be key:value
2019-09-02 14:07:26 +01:00
jesopo
9cc1ee98eb
Pass the content of a webpage to HTTPParsingException
2019-09-02 13:27:44 +01:00
jesopo
408b89aeb7
use \S+ for url regex (for non-ascii chars), use url_sanitize to catch <>
2019-09-02 13:25:48 +01:00
jesopo
9e43ea1599
add per-user fediverse setting
2019-09-02 08:51:56 +01:00
jesopo
19d8f0b497
store and expose reconnection timers on IRCBot.reconnections
2019-09-01 08:38:22 +01:00
jesopo
6a068777d6
v1.11.1 release
2019-08-30 19:10:17 +01:00
jesopo
722d6418c8
switch to using signal.setitimer so we can restore previous timer
2019-08-30 18:36:48 +01:00
jesopo
a57a06b1cc
switch utils.deadline to a context manager
2019-08-30 18:13:12 +01:00
jesopo
e97e3c8f30
add utils.deadline(func, seconds) to put hard limits on code exec time
2019-08-30 17:17:03 +01:00
jesopo
4bfb4c3200
Add IntRangeSetting, reorder OptionsSetting arg order
2019-08-30 14:40:54 +01:00
jesopo
2c812ccaaa
Logging.BitBotFormatter no longer uses the format it's given (always iso8601)
2019-08-30 13:46:21 +01:00
jesopo
58a4fd7e74
round microseconds up to a whole number and zero-pad to 3 chars
2019-08-30 13:46:00 +01:00
jesopo
d3ed08e336
Don't use 'Z' for "utc timezone" - actually parse timezone out of datetimes
2019-08-30 13:25:19 +01:00
jesopo
b037c3076f
v1.11.0 release
2019-08-15 13:45:54 +01:00
jesopo
61eeba0cb9
should only return a BufferLineMatch when we actually match
2019-08-15 13:45:04 +01:00
jesopo
195cb66e26
Update CHANGELOG.md, bump version to v1.11.0-rc3
2019-08-14 15:17:01 +01:00
jesopo
814236915b
forgot to update IRCBot.VERSION to v1.11.0-rc2
2019-08-13 15:11:43 +01:00
jesopo
03c7e8f066
return matching string from buffer.find() as most uses were redundantly regexing
2019-08-13 13:48:03 +01:00
jesopo
0f4a4d04a1
v1.11.0-rc1 release
2019-08-09 15:02:10 +01:00
jesopo
b6361ba4e3
Remove extraneous space
2019-08-07 14:00:56 +01:00
jesopo
20042edfd9
Allow bypass of content-type check in utils.http.request
2019-08-05 15:41:02 +01:00
jesopo
d093027431
not all HTTP responses have content-type
2019-08-02 17:33:16 +01:00
jesopo
dd8b276b35
pull priority out of kwargs in _hook
2019-07-30 15:24:13 +01:00
jesopo
27a6078f20
Revert "hook priority position should default to 0, not end"
...
This reverts commit ff0cc59507
2019-07-30 15:23:13 +01:00
jesopo
5ffe2941e8
Revert "we should be checking if the new hook has a higher priority"
...
This reverts commit b145e664e1
2019-07-30 15:23:05 +01:00
jesopo
b145e664e1
we should be checking if the new hook has a higher priority
2019-07-30 15:12:29 +01:00
jesopo
ff0cc59507
hook priority position should default to 0, not end
2019-07-30 15:09:34 +01:00
jesopo
793fc94ea3
source should have ':' appended
2019-07-28 20:55:54 +01:00
jesopo
ad8f66c00e
Remove debug print
2019-07-28 18:14:44 +01:00
jesopo
163e979f95
move socket creation to it's own function, send event just prior to .connect
2019-07-28 18:07:23 +01:00
jesopo
9827feeabd
'name' -> 'key' (copypaste fail)
2019-07-28 18:07:01 +01:00
jesopo
25b507e80c
Refactor hook kwargs to be stored as a list of tuples to support key duplicates
...
closes #108
2019-07-28 12:35:04 +01:00
jesopo
77dfc76591
switch to function/module magic being a single object
2019-07-26 11:58:06 +01:00
jesopo
b6844303cb
add set_throttle(lines, seconds) function to tweak throttle per-server
2019-07-23 17:46:22 +01:00
jesopo
4babce5357
Only fill throttle when buffer is empty
2019-07-22 13:14:07 +01:00
jesopo
304fcdaebb
gotta str()-ify kwargs before logging it
2019-07-15 14:27:55 +01:00
jesopo
4815c1612c
don't silently truncate at newline in IRCLine.ParsedLine.truncate()
2019-07-15 14:26:45 +01:00
Mikaela Suomalainen
2f18414cb7
src/EventManager.py: fix typo defualt -> default
2019-07-13 13:04:37 +03:00
jesopo
89c12e845a
add --remove-server/-R cli arg
2019-07-12 09:30:22 +01:00
jesopo
ab45c65fb4
'self.channels' -> 'self._channels'
2019-07-11 14:22:33 +01:00
jesopo
02838e4802
Return new Timers
2019-07-09 11:16:34 +01:00
jesopo
32fa999c78
Allow Timer objects to be cancelled
2019-07-09 11:14:05 +01:00
jesopo
c19c6c0e14
asyncio.gather -> asyncio.wait (with timeout)
2019-07-08 14:50:11 +01:00
jesopo
469c725675
tell asyncio.gather which loop to use
2019-07-08 14:41:12 +01:00
jesopo
a1438abf66
close event loop when we're done with it (request_many())
2019-07-08 13:59:48 +01:00
jesopo
81c7af8ab5
Don't try/except async http exceptions
2019-07-08 13:51:02 +01:00
jesopo
ee0ec0eca1
switch request_many() to use asyncio.gather
2019-07-08 13:46:27 +01:00
jesopo
b62ba469d7
catch async exceptions in utils.http.request_many()
2019-07-08 13:18:59 +01:00
jesopo
078681eddf
add missing schema in utils.http.sanitise_url, use in rss.py
2019-07-08 12:54:06 +01:00
jesopo
ecb8364d0d
switch to using asyncio's event loop
2019-07-08 12:45:10 +01:00
jesopo
15e143fcff
implement utils.http.request_many as a tonado ioloop yield
2019-07-08 11:43:09 +01:00
jesopo
f9d5271762
"paramatered" -> "parametered"
2019-07-04 17:23:36 +01:00
jesopo
43c8ad0ff6
Add !editserver command
2019-07-04 13:15:51 +01:00
jesopo
3afbe5fd82
threading.Lock() around any access to _write_buffer
2019-07-04 06:56:05 +01:00
jesopo
76ab7935a0
return parsed modes from IRCChannel.parse_modes, pass through mode events
2019-07-03 08:13:28 +01:00
jesopo
637067c62c
url_validate() -> url_sanitise()
2019-07-02 14:15:49 +01:00
jesopo
534854127b
Add utils.http.url_validate() for best-effort url tidying
2019-07-02 14:10:18 +01:00
jan6
b6e9713a14
fixed capabilities
2019-07-01 14:41:47 +03:00
jesopo
19f716ab4c
Implemnt _options_factory for OptionsSetting to have dynamic options
2019-06-29 22:26:28 +01:00
jesopo
b4c762eb4e
Automatically format example text for OptionsSetting (showing options)
2019-06-29 21:33:26 +01:00
jesopo
7091860e54
Setting.__init__ call in OptionsSetting ctor needs self param
2019-06-28 23:27:56 +01:00
jesopo
756396d758
Implement utils.OptionsSetting
2019-06-28 23:26:42 +01:00
jesopo
ae9d099a41
Refactor set/channelset/serverset/botset in to 'utils.Setting' objects
2019-06-28 23:16:05 +01:00
jesopo
f9eb017466
message arg for HTTPWrongContentTypeException/HTTPParsingException
2019-06-28 23:01:21 +01:00
jesopo
97810db8df
Give descriptions to utils.http.HTTPException subclasses
2019-06-27 18:28:08 +01:00
jesopo
3c3f0449b6
add CaseInsensitiveDict.__contains__
2019-06-26 17:57:49 +01:00
jesopo
16d331dd43
add allow_redirects kwarg to utils.http.request()
2019-06-26 17:53:16 +01:00
jesopo
c5785a2d14
implement @utils.kwarg() magic, use it for command.regex hooks
2019-06-26 14:37:41 +01:00
jesopo
7060a0ac67
Update IRCBot except queue.Empty comment
2019-06-26 13:43:00 +01:00
jesopo
4bf5c4783b
Re-add EventManager check to prevent _call() outside main thread
2019-06-26 11:42:23 +01:00
jesopo
bb749b111f
Refactor EventManager to only hold hooks on a root object
2019-06-26 11:01:09 +01:00
jesopo
12181bfec6
Add system to have multiple url shorteners and chose which to use
2019-06-25 17:53:00 +01:00
jesopo
a1cc2e778c
Return 0 seconds if timeout seconds is negative
2019-06-24 17:22:08 +01:00
jesopo
ef0b3ec64c
move _check() after timeouted queue.get() so deadlines are met accurately
2019-06-23 20:50:40 +01:00
jesopo
dd1df4dc05
move get_poll_timeout() to event loop now that it is the one calling _check
2019-06-23 19:00:37 +01:00
jesopo
9673963a8d
shift _check() call to event loop function so read_loop needn't call trigger()
2019-06-23 18:53:25 +01:00
jesopo
102aa1dce1
v1.10.0 release
2019-06-23 14:45:56 +01:00
jesopo
afc287f020
Don't print BitBotPanic stacks when we don't need to
2019-06-23 10:01:15 +01:00
jesopo
7861210495
v1.10.0-rc2
2019-06-23 09:45:47 +01:00
jesopo
22574448da
until_read_timeout is a func - read_timed_out has always been returning false
2019-06-22 23:31:06 +01:00
jesopo
01bad3a76e
Don't needlessly call time.monotonic() when checking cache expirations
2019-06-22 23:30:15 +01:00
jesopo
7d49826b1a
v1.10.0-rc1
2019-06-22 22:41:17 +01:00
jesopo
f1abc5f10c
remove remove_own_mode( check - deferred_read.py better solves the issue.
...
the issue was getting a MODE line on irc.com prior to 001, thus we didn't know
what our nickname was thus we didn't know that the MODE was for us. not dying
when we saw +x was easy to do with this check but I think it's more correct to
actually parse that MODE after 001 so we know we have +x.
2019-06-22 22:23:05 +01:00
jesopo
3eac36d0f5
Restrict EventManager calls to the main thread
2019-06-22 12:21:44 +01:00
jesopo
a783e71ad4
return generated SentLine objects from IRCChannel.send_* functions
2019-06-21 18:25:21 +01:00
jesopo
90c90e5bbd
Implement dependency system for CAPs
2019-06-21 18:05:11 +01:00
jesopo
8a0d99f969
Adda system of aliases for CAPs, mostly for changeable draft specs and creating
...
dependence between moving specs
2019-06-21 17:15:46 +01:00
jesopo
bffc5b0ec4
WARN log when we try to remove a mode from ourselves that we didn't have
2019-06-21 11:16:53 +01:00
jesopo
01438f28f6
check we have a mode before trying to remove it
2019-06-21 11:14:10 +01:00
jesopo
e7b17f2ccd
give every ParsedLine a uuid4 .id for tracking purposes
2019-06-21 10:33:59 +01:00
jesopo
01ddc04bb9
fix hostmask_match_many type hinting
2019-06-20 17:58:49 +01:00
jesopo
b6e194a450
Implement hostmask_match_any, for more efficient matching of multiple hostmasks
2019-06-20 14:21:11 +01:00
