Cloud Storage Guide
Storage Types Comparison
| Type | AWS / GCP / Azure | Access Pattern | Best For |
|---|---|---|---|
| Object Storage | S3 / GCS / Azure Blob | HTTP GET/PUT by key | Images, videos, backups, data lake, static sites |
| Block Storage | EBS / Persistent Disk / Azure Disk | Mounted as filesystem (ext4/NTFS) | VM root disks, databases, high IOPS workloads |
| File Storage (NFS) | EFS / Filestore / Azure Files | POSIX filesystem, shared mount | Shared config, CMS files, legacy lift-and-shift |
| Archive Storage | S3 Glacier / Cloud Archive | Asynchronous retrieval (hours) | Compliance, cold backups, long-term retention |
S3 API Operations
# AWS CLI
aws s3 mb s3://my-bucket --region us-east-1
# Upload / download
aws s3 cp file.jpg s3://my-bucket/images/file.jpg
aws s3 cp s3://my-bucket/images/file.jpg ./file.jpg
aws s3 sync ./local-folder s3://my-bucket/folder/ --delete
# List / remove
aws s3 ls s3://my-bucket/ --recursive --human-readable
aws s3 rm s3://my-bucket/old-file.txt
aws s3 rm s3://my-bucket/folder/ --recursive
# Presigned URL (time-limited access without credentials)
aws s3 presign s3://my-bucket/private-doc.pdf --expires-in 3600
# Enable versioning
aws s3api put-bucket-versioning \
--bucket my-bucket \
--versioning-configuration Status=Enabled
# Set lifecycle rule (transition to IA after 30d, Glacier after 90d)
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.json
# Block public access (best practice)
aws s3api put-public-access-block \
--bucket my-bucket \
--public-access-block-configuration \
"BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
Presigned URL Pattern (Go)
// Generate presigned upload URL (user uploads directly to S3)
func GeneratePresignedUploadURL(bucket, key string) (string, error) {
client := s3.NewPresignClient(s3Client)
req, err := client.PresignPutObject(context.Background(),
&s3.PutObjectInput{
Bucket: &bucket,
Key: &key,
ContentType: aws.String("image/jpeg"),
},
s3.WithPresignExpires(15*time.Minute),
)
if err != nil {
return "", err
}
return req.URL, nil
}
// CORS config (allow browser direct upload)
// {
// "CORSRules": [{
// "AllowedOrigins": ["https://myapp.com"],
// "AllowedMethods": ["PUT", "POST"],
// "AllowedHeaders": ["Content-Type"],
// "MaxAgeSeconds": 3000
// }]
// }