Brion Vibber [Fri, 22 Jan 2010 20:35:05 +0000 (12:35 -0800)]
Fix for stuck queue messages: wrap processing in stomp transactions so our lack of an ACK if PHP dies actually triggers redelivery.
Previously, messages once delivered would just get stuck in the queue seemingly forever if they never got ACKed.
Note this could lead to partial duplication, for instance if the OMB or Twitter queue handlers die after 1/2 of the outgoing sends.
Recommendations:
* catch exceptions more aggressively within queue handlers (so only PHP fatal errors are likely to kill in the middle)
* for processing that involves sending to multiple clients, consider a second queue similar to the XMPP output, eg for OMB:
- first queue gets delivery list and builds message data, enqueueing it for each target address
- second queue can handle each individual outgoing message (and attempt redelivery etc separately)
This would also protect better against a recurring error preventing delivery in the second part, and could spread out any slow sends over multiple threads.
Brion Vibber [Fri, 22 Jan 2010 00:42:50 +0000 (16:42 -0800)]
XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
Brion Vibber [Fri, 22 Jan 2010 00:42:50 +0000 (16:42 -0800)]
XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
Brion Vibber [Fri, 22 Jan 2010 00:27:08 +0000 (00:27 +0000)]
Quick hack to avoid breaking with geonames off when there's some old cookie state. This code's a little rough and tumble; any breakage halts JS execution and leaves the spinner going and no message submitted.
Craig Andrews [Sat, 16 Jan 2010 03:28:43 +0000 (22:28 -0500)]
Allow for instances as well as class names to be passed as queue handlers and iomanagers.
Introduce IoManager::GLOBAL_SINGLE_ONLY which indicates that only one instance of this iomanager will be run, regardless of how many threads/processes and sites there are.
Sarven Capadisli [Wed, 20 Jan 2010 17:32:24 +0000 (18:32 +0100)]
Updated notice item view where a) notice text no longer wraps around
(under author's photo) b) supplemental notice content and options will
start right under notice text.
Brion Vibber [Fri, 15 Jan 2010 21:06:21 +0000 (13:06 -0800)]
Fix for background recalculation of groups; needs to get Group objects not IDs; also don't list any groups for repeats to match saveGroups behavior
todo: merge calculation portion with saveGroups so they don't get out of sync
Brion Vibber [Fri, 15 Jan 2010 18:01:50 +0000 (10:01 -0800)]
Fix since_id and max_id API parameters for inbox-based loads; was failing if the exact id mentioned wasn't present in the inbox (or had been trimmed out)
Brion Vibber [Fri, 15 Jan 2010 18:01:50 +0000 (10:01 -0800)]
Fix since_id and max_id API parameters for inbox-based loads; was failing if the exact id mentioned wasn't present in the inbox (or had been trimmed out)
Brion Vibber [Thu, 14 Jan 2010 23:32:37 +0000 (15:32 -0800)]
temporary --skip-xmpp flag on queuedaemon.php, allows to run queue daemons but skip subscription to xmpp-based queues
(still working on making these behave gracefully when server is down)
projects / quix0rs-gnu-social.git / log