{
- "name": "friendica-addons/s3_storage",
- "description": "Adds the possibility to use S3 as a selectable storage backend",
- "type": "friendica-addon",
- "authors": [
- {
- "name": "Philipp Holzer",
- "email": "admin@philipp.info",
- "homepage": "https://blog.philipp.info",
- "role": "Developer"
+ "name": "friendica-addons/s3_storage",
+ "description": "Adds the possibility to use S3 as a selectable storage backend",
+ "type": "friendica-addon",
+ "authors": [
+ {
+ "name": "Philipp Holzer",
+ "email": "admin@philipp.info",
+ "homepage": "https://blog.philipp.info",
+ "role": "Developer"
+ }
+ ],
+ "require": {
+ "php": ">=7.0",
+ "akeeba/s3": "^2.0"
+ },
+ "license": "3-clause BSD license",
+ "config": {
+ "optimize-autoloader": true,
+ "autoloader-suffix": "S3StorageAddon",
+ "preferred-install": "dist"
}
- ],
- "require": {
- "php": ">=7.0",
- "akeeba/s3": "^2.0"
- },
- "license": "3-clause BSD license",
- "config": {
- "optimize-autoloader": true,
- "autoloader-suffix": "S3StorageAddon",
- "preferred-install": "dist"
- }
}
"packages": [
{
"name": "akeeba/s3",
- "version": "2.3.1",
+ "version": "2.3.2",
"source": {
"type": "git",
"url": "https://github.com/akeeba/s3.git",
- "reference": "7f5b3e929c93eb02ba24472560c0cbbef735aed9"
+ "reference": "452fbd3084f1cb581851a1602226224d29d586d4"
},
"dist": {
"type": "zip",
- "url": "https://api.github.com/repos/akeeba/s3/zipball/7f5b3e929c93eb02ba24472560c0cbbef735aed9",
- "reference": "7f5b3e929c93eb02ba24472560c0cbbef735aed9",
+ "url": "https://api.github.com/repos/akeeba/s3/zipball/452fbd3084f1cb581851a1602226224d29d586d4",
+ "reference": "452fbd3084f1cb581851a1602226224d29d586d4",
"shasum": ""
},
"require": {
"keywords": [
"s3"
],
- "time": "2023-09-26T11:40:10+00:00"
+ "time": "2024-02-19T10:08:06+00:00"
}
],
"packages-dev": [],
# Akeeba Amazon S3 Connector
-A compact, dependency-less Amazon S3 API client implementing the most commonly used features
+A compact, dependency-less Amazon S3 API client implementing the most commonly used features.
+
+This library is designed to work with Amazon S3 proper, as well as S3-compatible services such as but not limited to Wasabi, Google Storage, Synology C2, ExoScale etc.
## Why reinvent the wheel
```php
$configuration->setSSL(false);
-```
+```
+
+Caveat: HTTPS will only work if PHP can verify the TLS certificate of your endpoint. This may not be the case when using a local testing service (e.g. LocalStack), or for some buckets with dots in their names. Moreover, if you are on Windows, do note that neither PHP comes with a Certification Authority cache, nor is there a system-wide CA cache; you'll have to [download](https://curl.se/docs/caextract.html) it and configure PHP, or use [composer/ca-bundle](https://packagist.org/packages/composer/ca-bundle) in your `composer.json` file.
### Custom endpoint
'nyc3'
);
$configuration->setEndpoint('nyc3.digitaloceanspaces.com');
+$configuration->setRegion('nyc3');
+$configuration->setSignatureMethod('v4');
$connector = new \Akeeba\S3\Connector($configuration);
```
$connector = new \Akeeba\S3\Connector($configuration);
```
+Caveat: Setting the endpoint resets the signature version and region. This is why you need to set them _a second time_, after setting the endpoint, as seen in the first example above.
+
### Legacy path-style access
The S3 API calls made by this library will use by default the subdomain-style access. That is to say, the endpoint will be prefixed with the name of the bucket. For example, a bucket called `example` in the `eu-west-1` region will be accessed using the endpoint URL `example.s3.eu-west-1.amazonaws.com`.
-If you have buckets with characters that are invalid in the context of DNS (most notably dots and uppercase characters) this will fail. You will need to use the legacy path style instead. In this case the endpoint used is the generic region specific one (`s3.eu-west-1.amazonaws.com` in our example above) and the API URL will be prefixed with the bucket name.
+If you have buckets with characters that are invalid in the context of DNS (most notably dots and uppercase characters) this will fail. You will need to use the legacy path style instead. In this case the endpoint used is the generic region specific one (`s3.eu-west-1.amazonaws.com` in our example above), and the API URL will be prefixed with the bucket name.
You need to do:
```php
$configuration->setUseLegacyPathStyle(true);
```
-Caveat: this will not work with v2 signatures if you are using Amazon AWS S3 proper. It will very likely work with the v2 signatures if you are using a custom endpoint, though.
+Caveats:
+* This will not work with v2 signatures if you are using Amazon AWS S3 proper. It will very likely work with the v2 signatures if you are using a custom endpoint, though.
+* This option has no effect on pre-authorised (pre-signed) URLs. Legacy path-style access is used for these URLs by default.
### Dualstack (IPv4 and IPv6) support
$connector->setUseDualstackUrl(true);
```
-Caveat: this option only takes effect if you are using Amazon S3 proper. It will _not_ have any effect with custom endpoints.
\ No newline at end of file
+Caveat: This option only takes effect if you are using Amazon S3 proper. It will _not_ have any effect with custom endpoints. DualStack support is deprecated by Amazon S3. We strongly advise you NOT to use it anymore.
+
+### Alternate Date Format
+
+By default, this library uses the standard date format `D, d M Y H:i:s O` which Amazon _incorrectly_ documents as "ISO 8601" (it's not, see the [ISO 8601 Wikipedia entry](https://en.wikipedia.org/wiki/ISO_8601) for reference). Most third party, Amazon S3-compatible services use the same and understand it just fine.
+
+A minority of services don't understand the GMT offset at the end of the date format, and instead need the format `D, d M Y H:i:s T`. You can set a flag to enable this behaviour like so:
+```php
+$configuration->setAlternateDateHeaderFormat(true);
+```
+
+Caveat: Enabling this flag breaks compatibility with S3 proper.
+
+### Using The HTTP Date Header Instead Of X-Amz-Date
+
+Amazon documents that you should be using the standard HTTP `Date` header, and only resort to using the `X-Amz-Date` header when using the standard header is impossible, e.g. when creating pre-authorised (signed) URLs, or when your HTTP library does not let you set the standard header.
+
+Unfortunately, some third party S3-compatible services such as Wasabi and ExoScale do _NOT_ support the standard `Date` header at all. Using it makes them falsely spit out a message about the signature being wrong. They are the reason why, by default, we are passing the request date and time using the `X-Amz-Date` header.
+
+If you are using a third party service which for any reason does not understand the `X-Amz-Date` header you need to set a flag which forces the use of the standard `Date` header like so:
+```php
+$configuration->setUseHTTPDateHeader(true);
+```
\ No newline at end of file
# Testing notes
+**⚠️ WARNING**: Running all tests across all services takes _hours_. It is recommended that you use the SignedURLs test as a quick validation tool across multiple services, then test against the `$standardTests` suite overnight.
+
## Against Amazon S3 proper
This is the _canonical_ method for testing this library since Amazon S3 proper is the canonical provider of the S3 API (and not all of its quirks are fully documented, we might add).
## Against [LocalStack](https://localstack.cloud)
-This method is very useful for development.
+This method is very useful for development. It is the most faithful implementation of S3, but it does have some minor quirks not present in the real thing as a result of it running inside a Docker container.
-Install LocalStack [as per their documentation](https://docs.localstack.cloud/getting-started/installation/).
+Install LocalStack [per its documentation](https://docs.localstack.cloud/getting-started/installation/), or using its Docker Desktop Extension.
You will also need to install [`awslocal`](https://github.com/localstack/awscli-local) like so:
```php
pip install awscli-local
```
-Start LocalStack e.g. `localstack start -d`
-
-Create a new bucket called `test` i.e. `awslocal s3 mk s3://test`
+Start LocalStack e.g. `localstack start -d` or via the Docker Desktop Extension.
-Copy `config.dist.php` to `config.php` and make the following changes:
-```php
- define('DEFAULT_ENDPOINT', 'localhost.localstack.cloud:4566');
- define('DEFAULT_ACCESS_KEY', 'ANYRANDOMSTRINGWILLDO');
- define('DEFAULT_SECRET_KEY', 'ThisIsAlwaysIgnoredByLocalStack');
- define('DEFAULT_REGION', 'us-east-1');
- define('DEFAULT_BUCKET', 'test');
- define('DEFAULT_SIGNATURE', 'v4');
- define('DEFAULT_PATH_ACCESS', true);
-```
+Create a new bucket called `test` i.e. `awslocal s3 mb s3://test`
-Note that single- and dualstack tests result in the same URLs for all S3-compatible services, including LocalStack. These tests are essentially duplicates in this use case.
+The `config.dist.php` already has a configuration for LocalStack. Yes, the access and secret key can be any random string.
## Against Wasabi
-Wasabi nominally supports v4 signatures, but their implementation is actually _non-canonical_, as they only read the date from the optional `x-amz-date` header, without falling back to the standard HTTP `Date` header. We have added a workaround for this behaviour which necessitates testing with it.
+Wasabi nominally supports v4 signatures, but their implementation is actually _non-canonical_, as they only read the date from the optional `x-amz-date` header, without falling back to the standard HTTP `Date` header. We have changed the behaviour of the library to always go through the X-Amz-Date header as a result. Hence, the need to test with Wasabi.
+
+The Wasabi configuration block looks like this:
-Just like with Amazon S3 proper, copy `config.dist.php` to `config.php` and enter the connection information to your Wasabi storage. You will also need to set up the custom endpoint like so:
```php
-define('DEFAULT_ENDPOINT', 's3.eu-central-2.wasabisys.com');
+ 'wasabi-v4' => [
+ 'access' => 'THE_ACCESS_KEY',
+ 'secret' => 'VERY_SECRET_MUCH_WOW',
+ 'region' => 'eu-central-2',
+ 'bucket' => 'bucketname',
+ 'signature' => 'v4',
+ 'dualstack' => false,
+ 'path_access' => true,
+ 'ssl' => true,
+ 'endpoint' => 's3.eu-central-2.wasabisys.com',
+ ],
+ 'wasabi-v2' => [
+ 'access' => 'THE_ACCESS_KEY',
+ 'secret' => 'VERY_SECRET_MUCH_WOW',
+ 'region' => 'eu-central-2',
+ 'bucket' => 'bucketname',
+ 'signature' => 'v2',
+ 'dualstack' => false,
+ 'path_access' => true,
+ 'ssl' => true,
+ 'endpoint' => 's3.eu-central-2.wasabisys.com',
+ ],
+
```
-**IMPORTANT!** The above endpoint will be different, depending on which region you've created your bucket in. The example above assumes the `eu-central-2` region. If you use the wrong region the tests _will_ fail!
+**❗Important**: The Endpoint and Region must match each other, and the region the bucket was crated in. In the example above, we have created a bucket in the `eu-central-2` region. If you use the wrong region and/or endpoint the tests _will_ fail!
## Against Synology C2
-Synology C2 is an S3-“compatible” storage service. It is not very “compatible” though, since they implemented Amazon's documentation of the v4 signatures instead of how the v4 signatures work in the real world (yeah, there's a very big difference). While Amazon S3 _in reality_ expects all dates to be formatted as per RFC1123, they document that they expect them to be formatted as per “ISO 8601” and they give their _completely wrong_ interpretation of what the “ISO 8601” format is. Synology did not catch that discrepancy, and they only expect the wrongly formatted dates which is totally NOT what S3 itself expects. Luckily, most third party implementations expect either format because they've caught the discrepancy between documentation and reality, therefore making it possible for us to come up with a viable workaround.
+Synology C2 is an S3-“compatible” storage service. It is not very “compatible” though, since they implemented Amazon's documentation of the v4 signatures instead of how the v4 signatures work in the real world (yeah, there's a very big difference). While Amazon S3 _in reality_ expects all dates to be formatted as per RFC1123, they document that they expect them to be formatted as per “ISO 8601”, and they give their _completely wrong_ interpretation of what the “ISO 8601” format is. Synology did not catch that discrepancy, and they only expect the wrongly formatted dates which is totally NOT what S3 itself expects. Luckily, most third party implementations expect either format because they've caught the discrepancy between documentation and reality, therefore making it possible for us to come up with a viable workaround.
And that's why we need to test with C2 as well, folks.
-Copy `config.dist.php` to `config.php` and enter the connection information to your Synology S3 service.
+The C2 config block looks like this:
-It is very important to note two things:
```php
-define('DEFAULT_ENDPOINT', 'eu-002.s3.synologyc2.net');
-define('DEFAULT_REGION', 'eu-002');
+ 'c2' => [
+ 'access' => 'THE_ACCESS_KEY',
+ 'secret' => 'VERY_SECRET_MUCH_WOW',
+ 'region' => 'eu-002',
+ 'bucket' => 'bucketname',
+ 'signature' => 'v4',
+ 'dualstack' => false,
+ 'path_access' => false,
+ 'ssl' => true,
+ 'endpoint' => 'eu-002.s3.synologyc2.net',
+ ],
+
```
+
The endpoint URL is given in the Synology C2 Object Manager, next to each bucket. Note the part before `.s3.`. This is the **region** you need to use with v4 signatures. They do not document this anywhere.
\ No newline at end of file
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
return static::signedURL($s3, $options, Acl::ACL_PUBLIC_READ);
}
+ public static function signedURLPublicObjectSpaces(Connector $s3, array $options): bool
+ {
+ return static::signedURL($s3, array_merge($options, [
+ 'spaces' => true
+ ]), Acl::ACL_PUBLIC_READ);
+ }
+
public static function signedURLPrivateObject(Connector $s3, array $options): bool
{
return static::signedURL($s3, $options, Acl::ACL_PRIVATE);
}
+ public static function signedURLPrivateObjectSpaces(Connector $s3, array $options): bool
+ {
+ return static::signedURL($s3, array_merge($options, [
+ 'spaces' => true
+ ]), Acl::ACL_PRIVATE);
+ }
+
private static function signedURL(Connector $s3, array $options, string $aclPrivilege): bool
{
+ $spaces = isset($options['spaces']) && boolval($options['spaces']);
$tempData = static::getRandomData(AbstractTest::TEN_KB);
$input = Input::createFromData($tempData);
- $uri = 'test.' . md5(microtime(false)) . '.dat';
+ $prefix = $spaces ? 'test file' : 'test';
+ $uri = $prefix . '.' . md5(microtime(false)) . '.dat';
$s3->putObject($input, $options['bucket'], $uri, $aclPrivilege);
$downloadURL = $s3->getAuthenticatedURL($options['bucket'], $uri, null, $options['ssl']);
+
+ echo "\n\tDownload URL: $downloadURL\n";
+
$downloadedData = @file_get_contents($downloadURL);
try
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
-// Custom Endpoint. The example below is for using LocalStack, see https://localstack.cloud/
-// define('DEFAULT_ENDPOINT', 'localhost.localstack.cloud:4566');
-// Default Amazon S3 Access Key
-define('DEFAULT_ACCESS_KEY', 'your s3 access key');
-// Default Amazon S3 Secret Key
-define('DEFAULT_SECRET_KEY', 'your secret key');
-// Default region for the bucket
-define('DEFAULT_REGION', 'us-east-1');
-// Default bucket name
-define('DEFAULT_BUCKET', 'example');
-// Default signature method (v4 or v2)
-define('DEFAULT_SIGNATURE', 'v4');
-// Use Dualstack unless otherwise specified?
-define('DEFAULT_DUALSTACK', false);
-// Use legacy path access by default?
-define('DEFAULT_PATH_ACCESS', false);
-// Should I use SSL by default?
-define('DEFAULT_SSL', true);
// Create the 2100 test files in the bucket?
define('CREATE_2100_FILES', true);
+/**
+ * Configure the connection options for S3 and S3-compatible services here. Use them in the $testConfigurations array.
+ */
+$serviceConfigurations = [
+ 's3-v4' => [
+ 'access' => 'AK0123456789BCDEFGHI',
+ 'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
+ 'region' => 'eu-west-1',
+ 'bucket' => 'mybucket',
+ 'signature' => 'v4',
+ 'dualstack' => false,
+ 'path_access' => false,
+ 'ssl' => false,
+ 'endpoint' => null,
+ ],
+ 's3-v2' => [
+ 'access' => 'AK0123456789BCDEFGHI',
+ 'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
+ 'region' => 'eu-west-1',
+ 'bucket' => 'mybucket',
+ 'signature' => 'v2',
+ 'dualstack' => false,
+ 'path_access' => false,
+ 'ssl' => false,
+ 'endpoint' => null,
+ ],
+ 'localstack-v4' => [
+ 'access' => 'KYLOREN',
+ 'secret' => 'BenSolo',
+ 'region' => 'us-east-1',
+ 'bucket' => 'test',
+ 'signature' => 'v4',
+ 'dualstack' => false,
+ 'path_access' => true,
+ 'ssl' => false,
+ 'endpoint' => 'localhost.localstack.cloud:4566',
+ ],
+ 'localstack-v2' => [
+ 'access' => 'KYLOREN',
+ 'secret' => 'BenSolo',
+ 'region' => 'us-east-1',
+ 'bucket' => 'test',
+ 'signature' => 'v2',
+ 'dualstack' => false,
+ 'path_access' => true,
+ 'ssl' => false,
+ 'endpoint' => 'localhost.localstack.cloud:4566',
+ ],
+];
+
+/**
+ * Test EVERYTHING.
+ */
+$allTheTests = [
+ 'BucketsList',
+ 'BucketLocation',
+ 'HeadObject',
+ 'ListFiles',
+ 'SmallFiles',
+ 'SmallInlineFiles',
+ 'SmallFilesNoDelete',
+ 'SmallFilesOnlyUpload',
+ 'SmallInlineFilesNoDelete',
+ 'SmallInlineFilesOnlyUpload',
+ 'SmallInlineXMLFiles',
+ 'BigFiles',
+ 'Multipart',
+ 'StorageClasses',
+ 'SignedURLs',
+];
+
+if (CREATE_2100_FILES)
+{
+ $allTheTests[] = 'ListThousandsOfFiles';
+}
+
/**
* Tests for standard key pairs allowing us to read, write and delete
*
$standardTests = [
'BucketsList',
'BucketLocation',
- 'SmallFiles',
'HeadObject',
+ 'ListFiles',
+ 'SmallFiles',
'SmallInlineFiles',
'SmallInlineXMLFiles',
- 'SignedURLs',
- 'StorageClasses',
- 'ListFiles',
'BigFiles',
'Multipart',
+ 'StorageClasses',
+ 'SignedURLs',
];
/**
* - Buckets with international letters
* - Access from within EC2
*/
- 'Global key, v4, DNS, single stack' => [
- 'configuration' => [
- 'signature' => 'v4',
- 'dualstack' => false,
- 'path_access' => false,
- ],
+
+ // Amazon S3, v2 signatures
+ 'S3, v2, subdomain, single stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v2'],
+ [
+ 'dualstack' => false,
+ 'path_access' => false,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
- 'Global key, v4, DNS, dual stack' => [
- 'configuration' => [
- 'signature' => 'v4',
- 'dualstack' => true,
- 'path_access' => false,
- ],
+ 'S3, v2, subdomain, dual stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v2'],
+ [
+ 'dualstack' => true,
+ 'path_access' => false,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
- 'Global key, v4, path, single stack' => [
- 'configuration' => [
- 'signature' => 'v4',
- 'dualstack' => false,
- 'path_access' => true,
- ],
+ 'S3, v2, path, single stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v2'],
+ [
+ 'dualstack' => false,
+ 'path_access' => true,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
- 'Global key, v4, path, dual stack' => [
- 'configuration' => [
- 'signature' => 'v4',
- 'dualstack' => true,
- 'path_access' => true,
- ],
+ 'S3, v2, path, dual stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v2'],
+ [
+ 'dualstack' => true,
+ 'path_access' => true,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
- 'Global key, v2, DNS, single stack' => [
- 'configuration' => [
- 'signature' => 'v2',
- 'dualstack' => false,
- 'path_access' => false,
- ],
+ // Amazon S3, v4 signatures
+ 'S3, v4, subdomain, single stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v4'],
+ [
+ 'dualstack' => false,
+ 'path_access' => false,
+ ]
+ ),
+ 'tests' => $standardTests,
+ 'skip' => false,
+ ],
+ 'S3, v4, subdomain, dual stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v4'],
+ [
+ 'dualstack' => true,
+ 'path_access' => false,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
- 'Global key, v2, DNS, dual stack' => [
- 'configuration' => [
- 'signature' => 'v2',
- 'dualstack' => true,
- 'path_access' => false,
- ],
+ 'S3, v4, path, single stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v4'],
+ [
+ 'dualstack' => false,
+ 'path_access' => true,
+ ]
+ ),
'tests' => $standardTests,
+ 'skip' => false,
],
+ 'S3, v4, path, dual stack' => [
+ 'configuration' => array_merge(
+ $serviceConfigurations['s3-v4'],
+ [
+ 'dualstack' => true,
+ 'path_access' => true,
+ ]
+ ),
+ 'tests' => $standardTests,
+ 'skip' => false,
+ ],
+
+ // LocalStack
+ 'LocalStack, V2 (always path access, single stack)' => [
+ 'configuration' => $serviceConfigurations['localstack-v2'],
+ 'tests' => $allTheTests,
+ 'skip' => false,
+ ],
+
+ 'LocalStack, V4 (always path access, single stack)' => [
+ 'configuration' => $serviceConfigurations['localstack-v4'],
+ 'tests' => $allTheTests,
+ 'skip' => false,
+ ],
+
+ /**
+ * In the real config file we also have tests running on:
+ *
+ * - Wasabi, v2, path-style access.
+ * - Wasabi, v4, path-style access.
+ * - Synology C2, subdomain access.
+ *
+ * See [[NOTES.md]] for more information. If you have another environment you think we should test with please
+ * update NOTES.md and make a pull request, along with the reasoning behind it.
+ *
+ * There are known failures for some cases, notably LocalStack v4 (something is amiss?) and C2 (necessary flag is
+ * not yet supported by the minitest framework).
+ */
];
\ No newline at end of file
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
use Akeeba\S3\Configuration;
use Akeeba\S3\Connector;
-use Akeeba\S3\Input;
+
+/**
+ * The Miniature Test Framework For The Akeeba S3 Library
+ *
+ * This is a self-contained test-suite runner. Running minitest.php will execute all the tests against all
+ * configurations set up in the config.php file. When it's decked out with all real world examples this can take many
+ * hours to complete (it's not as "mini" as its name would like you to believe).
+ *
+ * Please read NOTES.md before proceeding and do keep in mind that some tests may fail for reasons outside the control
+ * of the library such as network conditions, whether PHP has a configured Certification Authority cache etc.
+ *
+ * As to why we didn't use Unit Tests: Elementary, dear Watson. Unit Tests are great when testing your code against a
+ * specification. In this case, the specification would be the Amazon S3 API documentation. Sounds great in theory, but
+ * not even Amazon itself works according to its own documentation, let alone the third party "S3-compatible" services
+ * which each one implements its own interpretation of that documentation. Therefore, slow-as-heck integration testing
+ * is the only way to do any kind of meaningful testing.
+ */
// Necessary for including the library
define('AKEEBAENGINE', 1);
echo "▶ " . $description . PHP_EOL;
echo str_repeat('〰', 80) . PHP_EOL . PHP_EOL;
+ if ($setup['skip'] ?? false)
+ {
+ echo "\t🤡 Skipping\n\n";
+
+ continue;
+ }
+
// Extract the configuration options
if (!isset($setup['configuration']))
{
}
$configOptions = array_merge([
- 'access' => DEFAULT_ACCESS_KEY,
- 'secret' => DEFAULT_SECRET_KEY,
- 'region' => DEFAULT_REGION,
- 'bucket' => DEFAULT_BUCKET,
- 'signature' => DEFAULT_SIGNATURE,
- 'dualstack' => DEFAULT_DUALSTACK,
- 'path_access' => DEFAULT_PATH_ACCESS,
- 'ssl' => DEFAULT_SSL,
+ 'access' => defined('DEFAULT_ACCESS_KEY') ? DEFAULT_ACCESS_KEY : null,
+ 'secret' => defined('DEFAULT_SECRET_KEY') ? DEFAULT_SECRET_KEY : null,
+ 'region' => defined('DEFAULT_REGION') ? DEFAULT_REGION : null,
+ 'bucket' => defined('DEFAULT_BUCKET') ? DEFAULT_BUCKET : null,
+ 'signature' => defined('DEFAULT_SIGNATURE') ? DEFAULT_SIGNATURE : null,
+ 'dualstack' => defined('DEFAULT_DUALSTACK') ? DEFAULT_DUALSTACK : null,
+ 'path_access' => defined('DEFAULT_PATH_ACCESS') ? DEFAULT_PATH_ACCESS : null,
+ 'ssl' => defined('DEFAULT_SSL') ? DEFAULT_SSL : null,
'endpoint' => defined('DEFAULT_ENDPOINT') ? constant('DEFAULT_ENDPOINT') : null,
], $setup['configuration']);
$s3Configuration->setUseLegacyPathStyle($configOptions['path_access']);
$s3Configuration->setSSL($configOptions['ssl']);
+ // Feature flags
+ if (isset($configOptions['alternateDateHeaderFormat']))
+ {
+ $s3Configuration->setAlternateDateHeaderFormat((bool) $configOptions['alternateDateHeaderFormat']);
+ }
+
+ if (isset($configOptions['useHTTPDateHeader']))
+ {
+ $s3Configuration->setUseHTTPDateHeader((bool) $configOptions['useHTTPDateHeader']);
+ }
+
+ if (isset($configOptions['preSignedBucketInURL']))
+ {
+ $s3Configuration->setPreSignedBucketInURL((bool) $configOptions['preSignedBucketInURL']);
+ }
+
// Create the connector object
$s3Connector = new Connector($s3Configuration);
[$className, $method] = $callableSetup;
echo " ⏱ Tearing down {$className}:{$method}…";
call_user_func($callableTeardown, $s3Connector, $configOptions);
- echo "\r Teared down {$className} " . PHP_EOL;
+ echo "\r Tore down {$className} " . PHP_EOL;
}
}
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
*/
protected $endpoint = 's3.amazonaws.com';
+ /**
+ * Should I use an alternative date header format (D, d M Y H:i:s T instead of D, d M Y H:i:s O) for non-Amazon,
+ * S3-compatible services?
+ *
+ * This is enabled by default.
+ *
+ * @var bool
+ */
+ protected $alternateDateHeaderFormat = true;
+
+ /**
+ * Should I use the standard HTTP Date header instead of the X-Amz-Date header?
+ *
+ * @var bool
+ */
+ protected $useHTTPDateHeader = false;
+
+ /**
+ * Should pre-signed URLs include the bucket name in the URL? Only applies to v4 signatures.
+ *
+ * Amazon S3 and most implementations need this turned off (default). LocalStack seems to not work properly with the
+ * bucket name as a subdomain, hence the need for this flag.
+ *
+ * @var bool
+ */
+ protected $preSignedBucketInURL = false;
+
/**
* Public constructor
*
{
$this->useDualstackUrl = $useDualstackUrl;
}
+
+ /**
+ * Get the flag for using an alternate date format for non-Amazon, S3-compatible services.
+ *
+ * @return bool
+ */
+ public function getAlternateDateHeaderFormat(): bool
+ {
+ return $this->alternateDateHeaderFormat;
+ }
+
+ /**
+ * Set the flag for using an alternate date format for non-Amazon, S3-compatible services.
+ *
+ * @param bool $alternateDateHeaderFormat
+ *
+ * @return void
+ */
+ public function setAlternateDateHeaderFormat(bool $alternateDateHeaderFormat): void
+ {
+ $this->alternateDateHeaderFormat = $alternateDateHeaderFormat;
+ }
+
+ /**
+ * Get the flag indicating whether to use the HTTP Date header
+ *
+ * @return bool Flag indicating whether to use the HTTP Date header
+ */
+ public function getUseHTTPDateHeader(): bool
+ {
+ return $this->useHTTPDateHeader;
+ }
+
+ /**
+ * Set the flag indicating whether to use the HTTP Date header.
+ *
+ * @param bool $useHTTPDateHeader Whether to use the HTTP Date header
+ *
+ * @return void
+ */
+ public function setUseHTTPDateHeader(bool $useHTTPDateHeader): void
+ {
+ $this->useHTTPDateHeader = $useHTTPDateHeader;
+ }
+
+ public function getPreSignedBucketInURL(): bool
+ {
+ return $this->preSignedBucketInURL;
+ }
+
+ public function setPreSignedBucketInURL(bool $preSignedBucketInURL): void
+ {
+ $this->preSignedBucketInURL = $preSignedBucketInURL;
+ }
}
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Authenticated (pre-signed) URLs are always made against the generic S3 region endpoint, not the bucket's
* virtual-hosting-style domain name. The bucket is always the first component of the path.
*
- * For example, given a bucket called foobar and an object baz.txt in it we are pre-signing the URL
+ * For example, given a bucket called foobar, and an object baz.txt in it, we are pre-signing the URL
* https://s3-eu-west-1.amazonaws.com/foobar/baz.txt, not
* https://foobar.s3-eu-west-1.amazonaws.com/foobar/baz.txt (as we'd be doing with v2 signatures).
*
* object is true. Naturally, the default behavior being virtual-hosting-style access to buckets, this flag is
* most likely **false**.
*
- * Therefore we need to clone the Configuration object, set the flag to true and create a Request object using
+ * Therefore, we need to clone the Configuration object, set the flag to true and create a Request object using
* the falsified Configuration object.
*
* Note that v2 signatures are not affected. In v2 we are always appending the bucket name to the path, despite
$newConfig->setUseLegacyPathStyle(true);
// Create the request object.
- $uri = str_replace('%2F', '/', rawurlencode($uri));
$request = new Request('GET', $bucket, $uri, $newConfig);
if ($query)
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
$this->headers['Date'] = gmdate('D, d M Y H:i:s O');
// S3-"compatible" services use a different date format. Because why not?
- if (strpos($this->headers['Host'], '.amazonaws.com') === false)
+ if ($this->configuration->getAlternateDateHeaderFormat() && strpos($this->headers['Host'], '.amazonaws.com') === false)
{
$this->headers['Date'] = gmdate('D, d M Y H:i:s T');
}
curl_setopt($curl, CURLOPT_URL, $url);
/**
- * Set the optional x-amz-date header for third party services.
+ * Set the optional x-amz-date header instead of the standard HTTP Date header.
*
- * Amazon S3 proper expects to get the date from the Date header. Third party services typically implement the
- * (wrongly) documented behaviour of using the x-amz-date header but, if it's missing, fall back to the Date
- * header. Wasabi does not fall back; it only uses the x-amz-date header which is why we have to set it here if
- * the request iss not made to Amazon S3 proper.
+ * Amazon S3 proper expects to get the date from the Date header. It also allows you to instead use the optional
+ * X-Amz-Date header as a means to resolve situations where your HTTP library does not let you control the
+ * standard Date header. In other words, it accepts both, it encourages the standard Date header, but it will
+ * give priority to the X-Amz-Date header to help you get out of a sticky situation.
+ *
+ * Unfortunately, third party services which claim to be "S3-compatible" are written by people with poor reading
+ * skills or, more likely, under unrealistic time constraints to deliver working code. They are implementing the
+ * date handling behaviout wrong. Instead of using the Date header unless the X-Amz-Date header is set, they
+ * **expect** to only ever see the X-Amz-Date header. If it's missing, they do not fall back to the standard
+ * Date header; they just spit out a message about the signature being wrong. Wasabi and ExoScale are two prime
+ * examples of that, and only when using v2 signatures.
+ *
+ * To avoid this problem, we are now defaulting to always using the X-Amz-Date header everywhere. If you want to
+ * revert to using the Date header with S3 proper please use setUseHTTPDateHeader(true) to your configuration
+ * object. In this case DO NOT set the X-Amz-Date header yourself, or you're going to have a *really* bad time.
*/
- $this->headers['x-amz-date'] = strpos($this->headers['Host'], '.amazonaws.com') !== false
- ? ''
- : (new \DateTime($this->headers['Date']))->format('Ymd\THis\Z');
+ if (!$this->configuration->getUseHTTPDateHeader())
+ {
+ $this->amzHeaders['x-amz-date'] = (new \DateTime($this->headers['Date']))->format('Ymd\THis\Z');
+
+ unset ($this->headers['Date']);
+ }
/**
* Remove empty headers.
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
$search = '/' . $bucket;
- if (strpos($uri, $search) === 0)
- {
- $uri = substr($uri, strlen($search));
- }
+ // This does not look right... The bucket name must be included in the URL.
+// if (strpos($uri, $search) === 0)
+// {
+// $uri = substr($uri, strlen($search));
+// }
$queryParameters = array_merge($this->request->getParameters(), [
'AWSAccessKeyId' => $accessKey,
// See http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth
if (isset($headers['Expires']))
{
- $headers['Date'] = $headers['Expires'];
+ if (isset($headers['Date']))
+ {
+ $headers['Date'] = $headers['Expires'];
+ }
+ else
+ {
+ $amzHeaders['x-amz-date'] = $headers['Expires'];
+ }
+
unset ($headers['Expires']);
$isPresignedURL = true;
$stringToSign = $verb . "\n" .
($headers['Content-MD5'] ?? '') . "\n" .
($headers['Content-Type'] ?? '') . "\n" .
- $headers['Date'] .
+ ($headers['Date'] ?? '') .
$amzString . "\n" .
$resourcePath;
// CloudFront only requires a date to be signed
if ($headers['Host'] == 'cloudfront.amazonaws.com')
{
- $stringToSign = $headers['Date'];
+ $stringToSign = $headers['Date'] ?? $amzHeaders['x-amz-date'] ?? '';
}
$amazonV2Hash = $this->amazonV2Hash($stringToSign);
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
$bucket = $this->request->getBucket();
$hostname = $this->getPresignedHostnameForRegion($region);
- if ($this->isValidBucketName($bucket))
+ if (!$this->request->getConfiguration()->getPreSignedBucketInURL() && $this->isValidBucketName($bucket))
{
$hostname = $bucket . '.' . $hostname;
}
// The query parameters are returned serialized; unserialize them, then build and return the URL.
$queryParameters = unserialize($serialisedParams);
- if ($this->isValidBucketName($bucket) && strpos($uri, '/' . $bucket) === 0)
+ // This should be toggleable
+ if (
+ !$this->request->getConfiguration()->getPreSignedBucketInURL()
+ && $this->isValidBucketName($bucket)
+ && strpos($uri, '/' . $bucket) === 0)
{
$uri = substr($uri, strlen($bucket) + 1);
}
}
// Get the credentials scope
- $signatureDate = new DateTime($headers['Date']);
+ $signatureDate = new DateTime($headers['Date'] ?? $amzHeaders['x-amz-date']);
$credentialScope = $signatureDate->format('Ymd') . '/' .
$this->request->getConfiguration()->getRegion() . '/' .
// The canonical URI is the resource path
$canonicalURI = $resourcePath;
- $bucketResource = '/' . $bucket;
+ $bucketResource = '/' . $bucket . '/';
$regionalHostname = ($headers['Host'] != 's3.amazonaws.com')
&& ($headers['Host'] != $bucket . '.s3.amazonaws.com');
+ /**
+ * Yet another special case for third party, S3-compatible services, when using pre-signed URLs.
+ *
+ * Given a bucket `example` and filepath `foo/bar.txt` the canonical URI to sign is supposed to be
+ * /example/foo/bar.txt regardless of whether we are using path style or subdomain hosting style access to the
+ * bucket.
+ *
+ * When calculating a pre-signed URL, the URL we will be accessing will be something to the tune of
+ * example.endpoint.com/foo/bar.txt. Amazon S3 proper allows us to use EITHER the nominal canonical URI
+ * /foo/bar.txt OR the /example/foo/bar.txt canonical URI for consistency. Some third party providers, like
+ * Wasabi, will choke on the former and complain about the signature being invalid.
+ *
+ * To address this issue we check if all the following conditions are met:
+ * - We are calculating a signature for a pre-signed URL.
+ * - The service is NOT Amazon S3 proper.
+ * - The domain name starts with the bucket name.
+ * In this case, and this case only, we set $regionalHostname to false. This triggers an if-block further down
+ * which strips the `/bucketName/` prefix from the canonical URI, converting it to `/`. Therefore, the canonical
+ * URI in the signature becomes the nominal URI we will be accessing in the bucket, solving the problem with
+ * those third party services.
+ */
+ // Figuring out whether it's a regional hostname DOES NOT work above if it's not AWS S3 proper. Let's fix that.
+ if ($isPresignedURL && strpos($headers['Host'], 'amazonaws.com') === false && !strpos($headers['Host'], $bucket . '.'))
+ {
+ $regionalHostname = false;
+ }
+
// Special case: if the canonical URI ends in /?location the bucket name DOES count as part of the canonical URL
// even though the Host is s3.amazonaws.com (in which case it normally shouldn't count). Yeah, I know, it makes
// no sense!!!
$regionalHostname = true;
}
- if (!$regionalHostname && (strpos($canonicalURI, $bucketResource) === 0))
+ if (!$regionalHostname && (strpos($canonicalURI, $bucketResource) === 0 || strpos($canonicalURI, substr($bucketResource, 0, -1)) === 0))
{
- if ($canonicalURI === $bucketResource)
+ if ($canonicalURI === substr($bucketResource, 0, -1))
{
$canonicalURI = '/';
}
else
{
- $canonicalURI = substr($canonicalURI, strlen($bucketResource));
+ $canonicalURI = substr($canonicalURI, strlen($bucketResource) - 1);
}
}
* headers if the request is made to a service _other_ than Amazon S3 proper.
*/
$dateToSignFor = strpos($headers['Host'], '.amazonaws.com') !== false
- ? $headers['Date']
+ ? (($headers['Date'] ?? null) ?: ($amzHeaders['x-amz-date'] ?? null) ?: $signatureDate->format('Ymd\THis\Z'))
: $signatureDate->format('Ymd\THis\Z');
$stringToSign = "AWS4-HMAC-SHA256\n" .
$endpoint = 's3.' . $region . '.amazonaws.com';
}
- $dualstackEnabled = $this->request->getConfiguration()->getDualstackUrl();
+ // As of October 2023, AWS does not consider DualStack signed URLs as valid. Whatever.
+ $dualstackEnabled = false && $this->request->getConfiguration()->getDualstackUrl();
// If dual-stack URLs are enabled then prepend the endpoint
if ($dualstackEnabled)
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
* Akeeba Engine
*
* @package akeebaengine
- * @copyright Copyright (c)2006-2023 Nicholas K. Dionysopoulos / Akeeba Ltd
+ * @copyright Copyright (c)2006-2024 Nicholas K. Dionysopoulos / Akeeba Ltd
* @license GNU General Public License version 3, or later
*/
[
{
"name": "akeeba/s3",
- "version": "2.3.1",
- "version_normalized": "2.3.1.0",
+ "version": "2.3.2",
+ "version_normalized": "2.3.2.0",
"source": {
"type": "git",
"url": "https://github.com/akeeba/s3.git",
- "reference": "7f5b3e929c93eb02ba24472560c0cbbef735aed9"
+ "reference": "452fbd3084f1cb581851a1602226224d29d586d4"
},
"dist": {
"type": "zip",
- "url": "https://api.github.com/repos/akeeba/s3/zipball/7f5b3e929c93eb02ba24472560c0cbbef735aed9",
- "reference": "7f5b3e929c93eb02ba24472560c0cbbef735aed9",
+ "url": "https://api.github.com/repos/akeeba/s3/zipball/452fbd3084f1cb581851a1602226224d29d586d4",
+ "reference": "452fbd3084f1cb581851a1602226224d29d586d4",
"shasum": ""
},
"require": {
"ext-simplexml": "*",
"php": ">=7.1.0 <8.4"
},
- "time": "2023-09-26T11:40:10+00:00",
+ "time": "2024-02-19T10:08:06+00:00",
"type": "library",
"installation-source": "dist",
"autoload": {
projects / friendica-addons.git / commitdiff