Brion Vibber [Thu, 27 Jan 2011 20:27:31 +0000 (12:27 -0800)]
Session GC fix: save session.modified field as UTC so our comparisons work.
Had to tweak statusnet.ini to remove the DB_DATAOBJECT_MYSQLTIMESTAMP bitfield constant on session.modified; while it sounds like a useful and legit setting, it actually just means that DB_DataObject silently fails to pass through any attempts to explicitly set the value. As a result, MySQL does its default behavior which is to insert the current *LOCAL* time, which is useless.
This was leading to early GC west of GMT, or late GC east of it. Early GC could at worst destroy all live sessions (whoever's session *triggered* GC is fine, as the session then gets saved right back.)
These affect twitterstatusfetcher.php on all 32-bit installs and some 64-bit installs (depending on whether the version of the JSON library reads the large numbers as long or double internally). 64-bit bug is harder to see as it tends to manifest as off-by-one due to losing a bit of precision off the end.
Brion Vibber [Fri, 7 Jan 2011 22:48:40 +0000 (14:48 -0800)]
Use ReflectionFunction to check for a present-but-disabled dl() function instead of manually parsing the disable_functions php.ini setting.
We were checking the list as comma-delimited (per the description of it as comma-delimited), but in fact spaces are also accepted, and who knows what else.
Brion Vibber [Wed, 5 Jan 2011 20:26:20 +0000 (12:26 -0800)]
Fixes for RegisterThrottle plugin: alt registration methods (OpenID, FBConnect, Twitter) weren't triggering the throttle check or recording of IPs.
Added StartRegistrationTry/EndRegistrationTry calls into those three, and moved the actual recording hook to EndUserRegister which is guaranteed to be called from User::register (so we don't need to worry about other auth methods forgetting to call the other UI-code hooks).
Brion Vibber [Tue, 28 Dec 2010 19:34:02 +0000 (11:34 -0800)]
Prevent group creation by silenced users.
* adds Right::CREATEGROUP
* logic in Profile::hasRight() checks for silencing
* NewgroupAction checks for the permission before letting you see or process the form in the UI
* User_group::register() logic does a low-level check on the specified initial group admin, and rejects creation if that user doesn't have the right; guaranteeing that API methods etc will also have this restriction applied sensibly.
Brion Vibber [Mon, 13 Dec 2010 20:12:22 +0000 (12:12 -0800)]
TwitterBridge: partial merge of id_str usage from 0.9.x for improved 32-bit and pre-5.2.10 compatibility. (on 64-bit in 5.2.6 we can pull the integer IDs, but silently lose some precision on the end.)
Fixes for Twitter bridge breakage on 32-bit servers. New "Snowflake" 64-bit IDs have become too big to fit in the integer portion of double-precision floats, so to reliably use these IDs we need to pull the new string form now.
Machines with 64-bit PHP installation should have had no problems (except on Windows, where integers are still 32 bits)
Conflicts:
plugins/TwitterBridge/twitterimport.php <- as this hasn't been broken out, the import code is NOT FULLY UPDATED HERE.
Brion Vibber [Fri, 10 Dec 2010 22:08:36 +0000 (22:08 +0000)]
Workaround for locally-handled sessions breaking on PHP 5.3 with APC enabled.
Big thanks to the folks at http://pecl.php.net/bugs/bug.php?id=16745 for the secret juju!
Classes were being torn down before session save handlers got called at the end of the request, which exploded with complaints about being unable to find various classes.
Registering a shutdown function lets us explicitly close out the session before everything gets torn down.
Brion Vibber [Mon, 3 May 2010 23:49:59 +0000 (16:49 -0700)]
extlibs updates: PEAR::Mail to 1.2.0, PEAR::Net_SMTP to 1.4.2 (need to go together as a pair)
PEAR::Mail updated to 1.2.0 from 1.1.4, fixes deprecation warnings on PHP 5.3, as well as:
1.2.0:
• QA release - stable.
• Updated minimum dependencies (Net_SMTP, PEAR, PHP)
• Doc Bug #15620 Licence change to BSD
• Bug #13659 Mail parse error in special condition
• Bug #16200 - Security hole allow to read/write Arbitrary File
_hasUnclosedQuotes() doesn't properly handle a double slash before an end quote (slusarz@curecanti.org, Bug #9137).
• Make sure Net_SMTP is defined when calling getSMTPObject() directly (slusarz@curecanti.org, Bug #13772).
• Add addServiceExtensionParameter() to the SMTP driver (slusarz@curecanti.org, Bug #13764).
• Add a method to obtain the Net_SMTP object from the SMTP driver (slusarz@curecanti.org, Bug #13766).
PEAR::Net_SMTP updated to 1.4.2 from 1.3.1, needed to support updated PEAR::Mail:
1.4.2:
• Fixing header string quoting in data(). (Bug #17199)
1.4.1:
• The auth() method now includes an optional $tls parameter that determines whether or not TLS should be attempted (if supported by the PHP runtime and the remote SMTP server). This parameter defaults to true. (Bug #16349)
• Header data can be specified separately from message body data by passing it as the optional second parameter to ``data()``. This is especially useful when an open file resource is being used to supply message data because it allows header fields (like *Subject:*) to be built dynamically at runtime. (Request #17012)
1.4.0:
• The data() method now accepts either a string or a file resource containing the message data. (Request #16962)
1.3.4:
• All Net_Socket write failures are now recognized. (Bug #16831)
1.3.3:
• Added getGreeting(), for retrieving the server's greeting string. (Request #16066) [needed for PEAR::Mail]
• We no longer attempt a TLS connection if we're already using a secure socket. (Bug #16254)
• You can now specify a debug output handler via setDebug(). (Request #16420)
1.3.2:
• TLS connection only gets started if no AUTH methods are sent. (Bug #14944)
Brion Vibber [Thu, 9 Dec 2010 01:39:04 +0000 (17:39 -0800)]
Tweak to PiwikAnalytics plugin to help browsers to pre-load piwik.js, may shave a little off load time.
Piwik's current default recommended JS for loading creates a <script> tag via document.write(). In addition to being generally evil, this means the browser doesn't know it's going to need piwik.js until that chunk of script gets executed... which can't happen until all scripts referenced *before* it have been loaded and executed.
The only reason for that bit of script though seems to be to pick 'http' or 'https' depending on the current page's scheme. This can be done more simply by using a protocol-relative link (eg "//piwik.status.net/piwik.js"), which the browser will resolve as appropriate. Since it's now sitting in the <script> tag, the browser's lookahead code will now see it and be able to start loading it while earlier things are parsing/executing.
May be better still to move to an asynchronous load after DOM-ready, but I'm not sure if that'll screw with the analytics code (eg, not being able to start things on the DOM-ready events since they're past).
Brion Vibber [Wed, 8 Dec 2010 22:54:02 +0000 (14:54 -0800)]
Mapstraction plugin: use minified sources for OpenLayers
The default full build of OpenLayers.js is 943kb as of 2.10; this gzips down to a couple hundred kb
but is still rather nasty, plus loading it off a remote host could slow things down.
Using a local copy let us cut down the size significantly by discarding unused features, and further
minification with yui-compressor shaves a bit more off. Cuts down to about 1/5 the size of the
original.
Also threw in a bundled & minified copy of the Mapstraction classes plus our usermap.js,
which covers the common case of using the default OpenLayers provider. This cuts out three
additional script loads, two of which weren't getting launched until after the mxn.js main
file got loaded.
Brion Vibber [Wed, 8 Dec 2010 22:52:26 +0000 (14:52 -0800)]
Create a bundled & minified JS file for Mapstraction's common case (using OpenLayers); this'll avoid waiting on additional script loads for mxn.core.js and mxn.openlayers.core.js, and removes the need to load usermap.js separately as well.
Brion Vibber [Wed, 8 Dec 2010 21:52:35 +0000 (13:52 -0800)]
Add stripped and minified local copy of OpenLayers 2.10, about 1/5 the size of the full version.
Included Makefile will recreate the OpenLayers.js using the statusnet.cfg strip configuration file
and yui-compressor to do some extra minification at the end. Requires fetching the OpenLayers
source download and dropping it in:
Evan Prodromou [Mon, 6 Dec 2010 22:05:32 +0000 (17:05 -0500)]
Make code-dependent cache entries more volatile
If a cache entry is dependent on the code that's running, upgrading
(or enabling/disabling plugins) can generate hard-to-track
inconsistencies.
This change adds a close-to-unique fingerprint of the running code to
some cache keys, so that if the fingerprint changes, the old values
are ignored and new values are used.
If the automated uniqueness fails, an administrator can add an extra
config value, $config['site']['build'], that's thrown into the key also.
Brion Vibber [Mon, 6 Dec 2010 20:39:09 +0000 (12:39 -0800)]
Added User::singleUserNickname() as (temporary?) fallback for single-user lookup as a workaround for site setup of 1user sites. We found that an external tool attempting to spin up StatusNet and then register the user would fail because StatusNet's router setup dies on being unable to find its single-user account, since the nickname is needed in setting up routing entries. This tweak will let it survive, using the configured setting as a fallback if it can't actually find the user account.
Brion Vibber [Tue, 30 Nov 2010 20:46:33 +0000 (12:46 -0800)]
Flush router cache when saving Twitter admin settings: adding/removing keys can enable/disable some actions. This avoids having users' Twitter settings unreachable after adding your keys to an installation with TwitterBridge on but not pre-configured.
Brion Vibber [Mon, 22 Nov 2010 19:10:10 +0000 (11:10 -0800)]
Fixes for delete_status_network.sh:
* add some sanity checking: abort on failures instead of plodding through
* add some progress / error output
* fetch the target database server name from the status_network entry and use that to target the DROP DATABASE
Note that database names and other overrides in status_network entry may still not be seen.
Brion Vibber [Fri, 19 Nov 2010 23:51:08 +0000 (15:51 -0800)]
Ticket #2724: gracefully handle attempts to delete or fave/unfave a remote Twitter notice if a failure occurs.
Most annoying error case being where the notice was already faved or deleted on Twitter! :)
Such errors will now just fail out and log a note to the syslog -- the rest of what we were doing will continue on unhindered, so you can still delete, favorite, etc and it just won't sync the info over in that case.
Brion Vibber [Fri, 19 Nov 2010 22:00:22 +0000 (14:00 -0800)]
Ticket #1987: support since_id on API notice search methods.
max_id is not yet implemented, as it'll need support added to the search backends. (since_id we get 'for free' by just cropping off the list, it'll do for now)
Brion Vibber [Fri, 19 Nov 2010 20:39:07 +0000 (12:39 -0800)]
Ticket #2441: fix deletion of avatars when a profile is deleted.
Code was doing a batch call to $avatar->delete() which fails to properly engage the file deletion code. Calling the existing profile->delete_avatars() function deletes them individually, which makes it all work nice again.
Brion Vibber [Fri, 19 Nov 2010 19:56:03 +0000 (11:56 -0800)]
Ticket #2899: clean up inbox/outbox DM form a bit:
- "To" drop-down list now defaults to showing "Select recipient:" instead of the first person on your list, reducing liklihood of accidentally sending a message to the wrong person.
- When there are no mutual subscribers to send to, instead of an empty list the list now shows 'No mutual subscribers.'
In both cases, attempting to send when the default is selected displays an error message.
I'm not disabling form elements in part because our themes right now don't show disabled button state correctly; we might want to tighten that up a bit more once fixed.
Brion Vibber [Thu, 18 Nov 2010 22:40:05 +0000 (14:40 -0800)]
scripts/deletegroup.php -- basic CLI script to delete a group by id or local nickname. Like deleteuser.php, this can be used in batch runs by providing the -y override.
Brion Vibber [Mon, 15 Nov 2010 23:34:12 +0000 (15:34 -0800)]
Stub RemoteprofileAction to show the standard profile header stuff for offsite users -- provides a way to get at the mod & block controls for remote users.
Brion Vibber [Mon, 15 Nov 2010 23:32:57 +0000 (15:32 -0800)]
some User -> Profile cleanup to help in adapting the profile page action to show stuff for remote users. Subscriptions, groups, roles, etc are all on profiles now so go ahead and use em.
Brion Vibber [Fri, 12 Nov 2010 21:06:41 +0000 (13:06 -0800)]
Fix for failure edge case in TwitterBridge outgoing repeat/retweets.
When the retweet failed with a 403 error (say due to it being a private tweet, which can't be retweeted) we would end up mishandling the return value from our internal error handling.
Instead of correctly discarding the message and closing out the queue item, we ended up trying to save a bogus twitter<->local ID mapping, which threw another exception and lead the queue system to re-run it.
- Fixed the logic check and return values for the retweet case in broadcast_twitter().
- Added doc comments explaining the return values on some functions in twitter.php
- Added check on Notice_to_status::saveNew() for empty input -- throw an exception before we try to actually insert into db. :)
Brion Vibber [Thu, 28 Oct 2010 19:58:30 +0000 (12:58 -0700)]
Kill a ping queue item if we get an error on loading up the notice's poster's profile, rather than letting the item be retried over and over as if it were a transitory error.
This shouldn't generally happen as it's an indicator of database inconsistency, but it's a condition we know happens.
Fields:
* action: case-normalized name of the action class we're acting on
* method: GET, POST, HEAD, etc
* ssl: Are we on HTTPS? 'yes' or 'no'
* query: Were we sent a query string? 'yes', 'no', or 'since_id' if the only parameter is a since_id
* cookie: Were we sent any cookies? 'yes' or 'no'
* auth: Were we sent an HTTP Authorization header? 'yes' or 'no'
* ifmatch: Were we sent an HTTP If-Match header for an ETag? 'yes' or 'no'
* ifmod: Were we sent an HTTP If-Modified-Since header? 'yes' or 'no'
* agent: User-agent string, to aid in figuring out what these things are
The most shared-cache-friendly requests will be non-SSL GET requests with no or very predictable
query parameters, no cookies, and no authorization headers. Private caching (eg within a supporting
user-agent) could still be friendly to SSL and auth'd GET requests.
We kind of expect that the most frequent hits from clients will be GETs for a few common timelines,
with auth headers, a since_id-only query, and no cookies. These should at least be amenable to
returning 304 matches for etags or last-modified headers with private caching, but it's very
possible that most clients won't actually think to save and send them. That would leave us expecting
to handle a lot of timeline since_id hits that return a valid API response with no notices.
At this point we don't expect to actually see if-match or if-modified-since a lot since most of our
API responses are marked as uncacheable; so even if we output them they're not getting sent back to
us.
Random subsampling can be enabled by setting the 'frequency' parameter smaller than 1.0:
addPlugin('ApiLogger', array(
'frequency' => 0.5 // Record 50% of API hits
));