Document migrating aws.s3.Bucket to aws.s3.BucketV2 #5444

t0yv0 · 2024-09-17T16:46:49Z

Description

Add a migration guide for moving from aws.s3.Bucket to aws.s3.BucketV2.

Fixes https://github.com/pulumi/home/issues/3630

Adding a new package?

If this pull request adds a new package:

t0yv0 · 2024-09-17T18:42:27Z

@interurban opening for early review, thanks a lot!

As an aside, I'm working to get examples formatted with the right <chooser> magic and I'm putting together a little script that updates the non-yaml sections automatically via pulumi convert. Would you all be interested in something like this or is there existing machinery to do this? Much appreciated.

github-actions · 2024-09-17T18:46:57Z

Your site preview for commit 6812b5d is ready! 🎉

http://registry--origin-pr-5444-6812b5d7.s3-website.us-west-2.amazonaws.com/registry.

github-actions · 2024-09-17T19:36:39Z

Your site preview for commit 95a25b7 is ready! 🎉

http://registry--origin-pr-5444-95a25b7f.s3-website.us-west-2.amazonaws.com/registry.

github-actions · 2024-09-17T19:46:51Z

Your site preview for commit f7e5595 is ready! 🎉

http://registry--origin-pr-5444-f7e5595e.s3-website.us-west-2.amazonaws.com/registry.

github-actions · 2024-09-17T20:11:27Z

Your site preview for commit 50358fc is ready! 🎉

http://registry--origin-pr-5444-50358fc1.s3-website.us-west-2.amazonaws.com/registry.

t0yv0 · 2024-09-17T21:15:00Z

Looks like something with choosers inside lists is not happy:

/home/runner/work/registry/registry/themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md:
Line 2417: Ordered list item prefix [Expected: 1; Actual: 6; Style: 1/2/3].
Line 2420: Ordered list item prefix [Expected: 2; Actual: 7; Style: 1/2/3].
Line 2422: Ordered list item prefix [Expected: 3; Actual: 8; Style: 1/2/3].

github-actions · 2024-09-18T02:52:43Z

Your site preview for commit f9a1f60 is ready! 🎉

http://registry--origin-pr-5444-f9a1f600.s3-website.us-west-2.amazonaws.com/registry.

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

flostadler · 2024-09-18T07:17:28Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+2. Perform `pulumi up` to replace (delete and re-create) the buckets in AWS.
+
+If replacement is not acceptable, it is possible to perform a manual migration with `pulumi import` (see Avoiding
+replacement).


We should be able to link to the section like this: [Avoiding Replacement](#avoiding-replacement). Not 100% sure though whether Hugo supports that syntax

It's been failing builds. Looks like this only works in GitHub markdown. @interurban might have some pointers here.

flostadler · 2024-09-18T07:20:41Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+
+
+
+As a bonus, the policy can now more easily refer to the concrete name of the bucket.


It might be worth explaining the benefit of not having cyclic deps in a bit more detail in the beginning. Gives some more insights into why this is a good thing for users.

It's a difficult sell. I tried adding a Context section at the bottom, but not sure it belongs, perhaps this sentence does not belong either. Primary motivation for users to do this is to get onto a supported version because they have to.

I've checked the style of https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade#s3-bucket-refactor FWIW it's very technical to-the-point and does not explain any design considerations as to why.

flostadler · 2024-09-18T07:27:05Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+This will only be available as an output. If your program specified this as an input, simply remove it. The input was
+ignored by prior versions of the provider and was exposed in error.
+
+### other inputs


How were the props with explicit instructions chosen? I guess they're the most used ones?

I'm wondering if we should have a single example with all options (might not be possible if some settings are mutually exclusive) and show how to migrate that instead.

I think I've just taken one of the more commonly used properties. It's not a bad idea, try running through this which might prove instructive.

flostadler · 2024-09-18T07:28:40Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+against the actual cloud account. While the details will vary depending on your use case, this procedure generally
+involves the following steps:
+
+- Find URNs for legacy Bucket Pulumi resources using `pulumi stack export`


We should be able to extract those with jq, right? Could be a nice addition to add that here

github-actions · 2024-09-18T18:30:16Z

Your site preview for commit a26fefd is ready! 🎉

http://registry--origin-pr-5444-a26fefd1.s3-website.us-west-2.amazonaws.com/registry.

github-actions · 2024-09-18T18:43:06Z

Your site preview for commit f172554 is ready! 🎉

http://registry--origin-pr-5444-f1725541.s3-website.us-west-2.amazonaws.com/registry.

t0yv0 · 2024-09-18T20:09:56Z

@flostadler I went ahead and took a larger bucket example for a spin through the import-based migration procedure. It worked but highlighted a few extra steps, this was useful.

For alternative upgrade procedures.

I have filed pulumi/pulumi-aws#4471 as this seems to have regressed.

I've also had a slack conversation with a user trying this today and filed pulumi/pulumi-aws#4470 - sounds like we could do a bit more possibly, depending on interest here.

github-actions · 2024-09-18T20:30:12Z

Your site preview for commit e7f99a0 is ready! 🎉

http://registry--origin-pr-5444-e7f99a0d.s3-website.us-west-2.amazonaws.com/registry.

georgberky · 2024-09-19T06:32:06Z

I've also had a slack conversation with a user trying this today and filed pulumi/pulumi-aws#4470 - sounds like we could do a bit more possibly, depending on interest here.

The confusing things for me were to find the old migration guide and

the full explanation being in a video instead of in text
the video showing a technique that my Typescript compiler forbids (redeclaring a variable)
the guide not being too clear that you have to declare a v2 resource next to the v1 resource instead of replacing v1 with v2 even though they have the same Pulumi resource name
the whole mechanism not working anymore
an IaC tool trying to delete my resource (luckily failing because the old bucket wasn't empty, but this doesn't raise confidence)

If the mechanism doesn't work anymore, remove the guide or clearly state on top that it stopped working with v12345 of Pulumi.

t0yv0 · 2024-09-19T13:15:13Z

Thanks for your feedback @georgberky that's a good point; we don't edit old blog content but we have a way to marking it obsolete and pointing to new material, I'll make sure we do this to https://www.pulumi.com/blog/announcing-v5.0.0-of-the-pulumi-aws-provider/#can-i-migrate-from-awss3bucket-to-awss3bucketv2

corymhall · 2024-09-19T14:23:24Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+   `BucketV2` inputs such as `policy` and `accelerationStatus` are to be replaced by side-by-side resources
+   `aws.s3.BucketPolicy` and `aws.s3.BucketAccelerateConfigurationV2`.
+
+2. Perform `pulumi up` to replace the buckets in AWS. **WARNING**: replacing the buckets will delete and re-create them,


I'm wondering if we should flip this and mention replacing buckets as the alternative. I think 99% of buckets won't be able to be replaced so we should just go with that as the default.

You're right.. that seems to be really true. I'll incorporate.

corymhall · 2024-09-19T14:25:39Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+import * as pulumi from "@pulumi/pulumi";
+import * as aws from "@pulumi/aws";
+
+const myBucket = new aws.s3.Bucket("my-bucket", {serverSideEncryptionConfiguration: {


Suggested change

const myBucket = new aws.s3.Bucket("my-bucket", {serverSideEncryptionConfiguration: {

const myBucket = new aws.s3.Bucket("my-bucket", {

serverSideEncryptionConfiguration: {

corymhall · 2024-09-19T14:34:55Z

themes/default/content/registry/packages/aws/how-to-guides/bucketv2-migration.md

+In situations when replacing the bucket in the AWS account is not acceptable, it is possible to perform a manual
+migration that changes Pulumi program and state to track buckets using the new resource without executing any changes
+against the actual cloud account. While the details will vary depending on your use case, this procedure generally
+involves the following steps:


I think as far as manual steps go this is not bad and as good as we can do 👍🏻

Document migrating aws.s3.Bucket to aws.s3.BucketV2

523d333

t0yv0 had a problem deploying to testing September 17, 2024 16:46 — with GitHub Actions Failure

Fix <Link fragments should be valid>

e5e581f

t0yv0 had a problem deploying to testing September 17, 2024 17:11 — with GitHub Actions Failure

Work around another x-ref build issue

6812b5d

t0yv0 temporarily deployed to testing September 17, 2024 18:13 — with GitHub Actions Inactive

t0yv0 requested review from cnunciato and interurban September 17, 2024 18:40

t0yv0 requested review from corymhall and flostadler September 17, 2024 18:42

Translate example 1 to yaml

c02c393

Render the first example

95a25b7

t0yv0 temporarily deployed to testing September 17, 2024 19:05 — with GitHub Actions Inactive

Add chooser for example #2

f7e5595

t0yv0 temporarily deployed to testing September 17, 2024 19:14 — with GitHub Actions Inactive

t0yv0 added 2 commits September 17, 2024 15:20

More choosers

e8e77e1

Choosers for more examples

a40589f

The rest of the choosers

50358fc

t0yv0 temporarily deployed to testing September 17, 2024 19:40 — with GitHub Actions Inactive

Final chooser

e442830

t0yv0 had a problem deploying to testing September 17, 2024 19:59 — with GitHub Actions Failure

Remove redundant heading

0e172c1

t0yv0 had a problem deploying to testing September 17, 2024 20:03 — with GitHub Actions Failure

Use un-numbered lists

f9a1f60

t0yv0 temporarily deployed to testing September 18, 2024 02:22 — with GitHub Actions Inactive

flostadler reviewed Sep 18, 2024

View reviewed changes

t0yv0 mentioned this pull request Sep 18, 2024

Consider building tooling to simplify migrating from Bucket to BucketV2 pulumi/pulumi-aws#4470

Open

PR feedback; add a context session

a26fefd

t0yv0 temporarily deployed to testing September 18, 2024 17:56 — with GitHub Actions Inactive

t0yv0 added 2 commits September 18, 2024 13:57

Simplify the grammar

c8a2dbf

Exampe corrections

f172554

t0yv0 temporarily deployed to testing September 18, 2024 18:10 — with GitHub Actions Inactive

t0yv0 added 2 commits September 18, 2024 16:02

Update migration notes on latest

820e95d

Regenerate choosers

e7f99a0

t0yv0 temporarily deployed to testing September 18, 2024 20:05 — with GitHub Actions Inactive

t0yv0 requested a review from flostadler September 18, 2024 20:07

t0yv0 marked this pull request as ready for review September 18, 2024 20:59

corymhall reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document migrating aws.s3.Bucket to aws.s3.BucketV2 #5444

Document migrating aws.s3.Bucket to aws.s3.BucketV2 #5444

t0yv0 commented Sep 17, 2024

t0yv0 commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

t0yv0 commented Sep 17, 2024

github-actions bot commented Sep 18, 2024

flostadler Sep 18, 2024

t0yv0 Sep 18, 2024

flostadler Sep 18, 2024

t0yv0 Sep 18, 2024

flostadler Sep 18, 2024

t0yv0 Sep 18, 2024

flostadler Sep 18, 2024

github-actions bot commented Sep 18, 2024

github-actions bot commented Sep 18, 2024

t0yv0 commented Sep 18, 2024

github-actions bot commented Sep 18, 2024

georgberky commented Sep 19, 2024 •

edited

Loading

t0yv0 commented Sep 19, 2024

corymhall Sep 19, 2024

t0yv0 Sep 19, 2024

corymhall Sep 19, 2024

corymhall Sep 19, 2024




		As a bonus, the policy can now more easily refer to the concrete name of the bucket.

	const myBucket = new aws.s3.Bucket("my-bucket", {serverSideEncryptionConfiguration: {
	const myBucket = new aws.s3.Bucket("my-bucket", {
	serverSideEncryptionConfiguration: {

Document migrating aws.s3.Bucket to aws.s3.BucketV2 #5444

Are you sure you want to change the base?

Document migrating aws.s3.Bucket to aws.s3.BucketV2 #5444

Conversation

t0yv0 commented Sep 17, 2024

Description

Adding a new package?

t0yv0 commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

github-actions bot commented Sep 17, 2024

t0yv0 commented Sep 17, 2024

github-actions bot commented Sep 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Sep 18, 2024

github-actions bot commented Sep 18, 2024

t0yv0 commented Sep 18, 2024

github-actions bot commented Sep 18, 2024

georgberky commented Sep 19, 2024 • edited Loading

t0yv0 commented Sep 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgberky commented Sep 19, 2024 •

edited

Loading