Skip to content

Draft: feature: Use pre-signed urls for uploading configuration versions

Filip Rutkowski requested to merge feature/presigned-configuration-version-urls into main

Since terraform 1.6.x the authorization header is no longer passed when uploading configuration versions. Using pre-signed URLs will allow to upload directly to the storage, bypassing terrarun.

This was a quick check if the pre-signed urls would fix my issue and they partially did. After my changes it locks up one step further when the configuration state is pending forever.

I'm not sure yet how to:

  • Create an alembic update to remove the blob from the DB
  • Update the configuration-version state from pending to uploaded

For the latter I have a few ideas:

  • Check in get_by_api_id if it's pending and if so check
  • Create another loop in worker. This will create unnecessary traffic, especially if the upload never happens. Would need to add a timeout after which we set the state to errored.
  • Minio webhook/S3 SNS Webhook. Slight dependency on an external tool, but we already lock in slightly with the pre-signed urls anyway.

@mjc any thoughts?

Edited by Matt

Merge request reports