AWS FileStorage NoSuchBucketException

nelsonf · July 2, 2024, 9:49pm

Hi Jmix team.

I have an application under development, which connects to MinIO (VERSION 2023-11-15T20:43:25Z) with the AWS File Storage component, up to version 2.2 of Jmix it works fine, but since we updated to the new 2.3 it generates a connection error to the bucket.

2024-07-02T16:09:23.428-05:00 ERROR 32876 --- [nio-8080-exec-2] io.jmix.awsfs.AwsFileStorage             : Error loading file from S3 storage

software.amazon.awssdk.services.s3.model.NoSuchBucketException: The specified bucket does not exist (Service: S3, Status Code: 404, Request ID: 17DE81900BDB6193, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125) ~[sdk-core-2.25.60.jar:na]

Attached is a complete trace of the error.
AWS File Storage error.txt (22.1 KB)

The connection parameters of the MinIO bucket are the following:

# Bucket
jmix.core.default-file-storage = s3
jmix.awsfs.access-key = xzkWuLh...
jmix.awsfs.secret-access-key = Zdd9KD9...
jmix.awsfs.region = us-east-1
jmix.awsfs.bucket = fs-mediesfera
jmix.awsfs.endpoint-url = http://localhost:9000

Any idea about this?

Saludos,

Nelson F.

gorbunkov · July 4, 2024, 9:30am

Hi,

In Jmix 2.3 we’ve upgraded AWS SDK version. The new version started using virtual host-style addressing when including an endpoint override (see AWS SDK docs).

In the next patch release we’ll add an application property to Jmix, that will force path-style bucket addressing. Here is the issue: Support path-style bucket addressing in AWS file storage · Issue #3448 · jmix-framework/jmix · GitHub

abdulmaliknajmiddino · July 6, 2024, 6:34am

Hi Jmix team

I have also encountered problem like above mentioned after updating jmix version to 2.3.0. But difference is that:

2024-07-06T11:25:36.255+05:00 ERROR 25316 — [nio-8080-exec-3] io.jmix.awsfs.AwsFileStorage : Error loading file from S3 storage
software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Certificate for <jobster4.jobster4.fra1.digitaloceanspaces.com> doesn’t match any of the subject alternative names: [*.fra1.digitaloceanspaces.com, fra1.digitaloceanspaces.com]
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:223) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:83) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:66) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:60) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52) ~[sdk-core-2.25.60.jar:na]
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:60) ~[aws-core-2.25.60.jar:na]
at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:5203) ~[s3-2.25.60.jar:na]
at io.jmix.awsfs.AwsFileStorage.openStream(AwsFileStorage.java:261) ~[jmix-awsfs-2.3.0.jar:na]

The connection parameters are the following:

jmix.core.default-file-storage = s3
jmix.awsfs.access-key = access-key
jmix.awsfs.secret-access-key = secret-access-key
jmix.awsfs.region = fra1
jmix.awsfs.bucket = jobster4
jmix.awsfs.chunk-size = 8192
jmix.awsfs.endpoint-url = https://jobster4.fra1.digitaloceanspaces.com

What solution can you provide for this?

gorbunkov · July 8, 2024, 1:48pm

Hi,

Unfortunately, we cannot guarantee that the AWS File Storage add-on will work with any S3 compatible storage with no issues. We haven’t check it with Digital Ocean Spaces yet.

As I see from DO documentation:

Due to an AWS-specific behavior in all versions of the SDK except Python 3, to successfully create a new bucket, you must specify an AWS region, such as us-east-1, in your configuration. This is because, when creating a bucket, the SDK sends an entirely different payload if a custom region is specified, which results in an error.
Specifying us-east-1 does not result in slower performance, regardless of your bucket’s location. The SDK checks the region for verification purposes but never sends the payload there. Instead, it sends the payload to the specified custom endpoint.

Please try setting the following application property: jmix.awsfs.region = us-east-1.

Also, how DO accesses buckets: do you use path-style or virtual-host style addressing? If it uses the path style, then the fix mentioned above may help you as well (we will introduce a new application property for forcing a path-style addressing).