Hi @n-batalha what happens if you initialize boto3 client directly and call upload_fileobj to put something in your bucket? The error suggests that this isn't something to do with prefect but with your access key.

@joshmeek the same credentials work to upload, I use it to store Prefect results in s3 and for a direct boto upload.

@joshmeek to be 100% sure I edited the reproducible example above to do an upload in the same bucket, works.

🤔 be sure that you aren't unintentionally reading AWS credentials from one of the other default locations that boto3 uses https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html

FWIW I ran your code exactly and had no issue uploading to a bucket:

➜ python s3storagetest.py
...my buckets printed here...
Result check: OK
[2020-08-06 11:51:08] INFO - prefect.S3 | Uploading say-hi/2020-08-06t11-51-08-712556-00-00 to my-bucket
Flow: https://cloud.prefect.io/tenant-name/flow/flow-id

@joshmeek good heads up. I believe the env vars take precedence so it should work for both.

The docs state (the part here) that the same setup is required as boto, but there is a bit going on in here:

• https://github.com/PrefectHQ/prefect/blob/master/src/prefect/environments/storage/s3.py#L201-L203

• prefect/src/prefect/utilities/aws.py

  Lines 10 to 63 in d31e5b0

  	def get_boto_client(
  	resource: str, credentials: dict = None, use_session: bool = False, **kwargs: Any
  	) -> "boto3.client":
  	"""
  	Utility function for loading boto3 client objects from a given set of credentials.
  	Args:
  	- resource (str): the name of the resource to retrieve a client for
  	- credentials (dict, optional): a dictionary of AWS credentials used to
  	initialize the Client; if not provided, will attempt to load the
  	Client using ambient environment settings
  	- use_session (bool, optional): a boolean specifying whether to load
  	this client using a session or not; defaults to `False`
  	- **kwargs (Any, optional): additional keyword arguments to pass to boto3
  	Returns:
  	- Client: an initialized and authenticated boto3 Client
  	"""
  	aws_access_key = None
  	aws_secret_access_key = None
  	aws_session_token = None
  	if credentials:
  	aws_access_key = credentials["ACCESS_KEY"]
  	aws_secret_access_key = credentials["SECRET_ACCESS_KEY"]
  	aws_session_token = credentials.get("SESSION_TOKEN")
  	else:
  	ctx_credentials = prefect.context.get("secrets", {}).get("AWS_CREDENTIALS", {})
  	aws_access_key = ctx_credentials.get("ACCESS_KEY")
  	aws_secret_access_key = ctx_credentials.get("SECRET_ACCESS_KEY")
  	aws_session_token = ctx_credentials.get("SESSION_TOKEN")
  	if use_session:
  	# see https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html?#multithreading-multiprocessing # noqa
  	session = boto3.session.Session()
  	return session.client(
  	resource,
  	aws_access_key_id=aws_access_key or kwargs.pop("aws_access_key_id", None),
  	aws_secret_access_key=aws_secret_access_key
  	or kwargs.pop("aws_secret_access_key", None),
  	aws_session_token=aws_session_token
  	or kwargs.pop("aws_session_token", None),
  	**kwargs
  	)
  	else:
  	return boto3.client(
  	resource,
  	aws_access_key_id=aws_access_key or kwargs.pop("aws_access_key_id", None),
  	aws_secret_access_key=aws_secret_access_key
  	or kwargs.pop("aws_secret_access_key", None),
  	aws_session_token=aws_session_token
  	or kwargs.pop("aws_session_token", None),
  	**kwargs
  	)

Removing ~/.aws had no impact (still fails), but removing the PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS env var allows the script above to run. I can see the env var above being picked up for the boto creds, as opposed to leaving it to boto entirely.

I am leaving this open as it should work when both PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS and the boto env vars are present?

Btw, debugging I can see that the PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS values are correctly parsed in here and the values themselves seem fine. They are pulled from the other env vars via:

export PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS='{"ACCESS_KEY": "'${AWS_ACCESS_KEY_ID}'", "SECRET_ACCESS_KEY": "'${AWS_SECRET_ACCESS_KEY}'"}'