Cellar, a S3-like object storage service

Cellar is a S3-compatible online file storage web service. Use it with your favorite S3 client, or download the s3cmd configuration file from the add-on dashboard in Clever Cloud console.

Creating a bucket

Cellar stores files in buckets. When you create a Cellar add-on, no bucket exists yet.

From Clever Cloud Console

With s3cmd

s3cmd mb s3://bucket-name

With AWS CLI

You can use the official AWS cli with Cellar. Configure the aws_access_key_id, aws_secret_access_key and endpoint.

aws configure set aws_access_key_id $CELLAR_ADDON_KEY_ID
aws configure set aws_secret_access_key $CELLAR_ADDON_KEY_SECRET

Global endpoint configuration isn’t available, so include the parameter each time you use the aws cli. Here’s an example to create a bucket:

aws s3api create-bucket --bucket myBucket --acl public-read --endpoint-url https://cellar-c2.services.clever-cloud.com

To simplify this, you may want to configure an alias like so:

alias aws="aws --endpoint-url https://cellar-c2.services.clever-cloud.com"

Managing your buckets

There are several ways to manage your buckets, find in this section a list of options.

Using S3 clients

Some clients allows you to upload files, list them, delete them, etc, like:

• Cyberduck
• Filestash
• MinIO
• S3 Browser
• WinSCP

This list isn’t exhaustive. Feel free to suggest other clients that you would like to see in this documentation.

Using s3cmd command line tools

s3cmd allows you to manage your buckets using its commands, after configuring it on your machine

s3cmd put --acl-public image.jpg s3://bucket-name

s3cmd ls s3://bucket-name

Custom domain

If you want to use a custom domain, for example cdn.example.com, you need to create a bucket named exactly like your domain:

s3cmd --host-bucket=cellar-c2.services.clever-cloud.com mb s3://cdn.example.com

Then, create a CNAME record on your domain pointing to cellar-c2.services.clever-cloud.com..

Using AWS SDK

To use cellar from your applications, you can use the AWS SDK. You only need to specify a custom endpoint (eg cellar-c2.services.clever-cloud.com).

// Load the AWS SDK for Node.js
const AWS = require('aws-sdk');

// Set up config
AWS.config.update({
  accessKeyId: '<cellar_key_id>', 
  secretAccessKey: '<cellar_key_secret>'
});

// Create S3 service object
const s3 = new AWS.S3({ endpoint: '<cellar_host>' });

// Create the parameters for calling createBucket
const bucketParams = {
  Bucket : '<my-bucket-name>',
  CreateBucketConfiguration: {
    LocationConstraint: ''
  }
};

// call S3 to create the bucket
s3.createBucket(bucketParams, function(err, data) {
  // handle results
});

// Call S3 to list the buckets
s3.listBuckets(function(err, res) {
  // handle results
});

/* In order to share access to access non-public files via HTTP, you need to get a presigned url for a specific key
 * the example above present a 'getObject' presigned URL. If you want to put a object in the bucket via HTTP,
 * you'll need to use 'putObject' instead.
 * see doc : https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property
 */
s3.getSignedUrl('getObject', {Bucket: '<YouBucket>', Key: '<YourKey>'})

<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>s3</artifactId>
  <version>2.21.35</version>
</dependency>

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.Bucket;
import software.amazon.awssdk.services.s3.model.ListObjectsRequest;

import java.net.URI;
import java.util.List;

public class CleverCloudCellarDemoApplication {

    // replace those values with your own keys, load them from properties or env vars
    private static final String CELLAR_HOST = "";
    private static final String CELLAR_KEY_ID = "";
    private static final String CELLAR_KEY_SECRET = "";

    public static void main(String[] args) {
        // initialize credentials with Cellar Key ID and Secret
        // you can also use `EnvironmentVariableCredentialsProvider` by setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars
        var credentialsProvider = StaticCredentialsProvider.create(AwsBasicCredentials.create(CELLAR_KEY_ID, CELLAR_KEY_SECRET));

        // create a client builder
        var s3ClientBuilder = S3Client.builder()
                // override the S3 endpoint with the cellar Host (starting with 'https://'
                .endpointOverride(URI.create(CELLAR_HOST))
                .credentialsProvider(credentialsProvider);

        // initialize the s3 client
        try (S3Client s3 = s3ClientBuilder.build()) {
            // list buckets
            List<Bucket> buckets = s3.listBuckets().buckets();
            buckets.forEach(bucket -> {
                // list bucket objects
                var listObjectsRequest = ListObjectsRequest.builder().bucket(bucket.name()).build();
                var objects = s3.listObjects(listObjectsRequest).contents();
                // handle results
            });

        }
    }
}

from boto.s3.key import Key
from boto.s3.connection import S3Connection
from boto.s3.connection import OrdinaryCallingFormat

apikey='<key>'
secretkey='<secret>'
host='<host>'

cf=OrdinaryCallingFormat()  # This mean that you _can't_ use upper case name
conn=S3Connection(aws_access_key_id=apikey, aws_secret_access_key=secretkey, host=host, calling_format=cf)

b = conn.get_all_buckets()
print(b)

"""
In order to share access to non-public files via HTTP, you need to get a presigned url for a specific key
the example above present a 'getObject' presigned URL. If you want to put a object in the bucket via HTTP,
you'll need to use 'putObject' instead.
see doc : https://docs.pythonboto.org/en/latest/ref/s3.html#boto.s3.bucket.Bucket.generate_url
"""
b[0].generate_url(60)

cellar:
  service: S3
  access_key_id: <%= ENV.fetch('CELLAR_ADDON_KEY_ID') %>
  secret_access_key: <%= ENV.fetch('CELLAR_ADDON_KEY_SECRET') %>
  endpoint: https://<%= ENV.fetch('CELLAR_ADDON_HOST') %>
  region: 'us-west-1'
  force_path_style: true
  bucket: mybucket

Policies

Cellar allows you to create policies to control the actions on your buckets. Find below two policies examples, and further documentation here.

Public bucket policy

You can upload all your objects with a public ACL, but you can also make your whole bucket publicly available in read mode. No one can access the write permission without authentication.

To set your bucket as public, you have to apply the following policy which you can save in a file named policy.json:

{
  "Id": "Policy1587216857769",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1587216727444",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::<bucket-name>/*",
      "Principal": "*"
    }
  ]
}

Replace the <bucket-name> with your bucket name in the policy file. Don’t change the Version field to the current date, keep it as is.

Now, you can set the policy to your bucket using s3cmd:

s3cmd setpolicy ./policy.json s3://<bucket-name>

💡 If you encounter errors, you might need to specify the configuration file path:

s3cmd setpolicy ./policy.json -c path/to/s3cfg.txt s3://<bucket-name>

All of your objects should now be publicly accessible.

If needed, you can delete this policy by using:

s3cmd delpolicy s3://<bucket-name>

The original ACL should apply to all of your objects after that.

User access

Cellar doesn’t natively support creating different user accesses for the same add-on. Granting access to your Cellar add-on grants full access to all of your buckets. To grant limited access to a bucket, do the following:

1. Create your main Cellar add-on (we’ll call it Cellar-1)
2. Download Cellar 1 s3cfg file
3. Create a second Cellar add-on (we’ll call it Cellar-2)
4. Get the ADDON ID from Cellar-2 dashboard (it should look like cellar_xxx)
5. Create a policy for Cellar-1 and inject the ADDON ID from Cellar-2 as the user.

Now, you can pass Cellar-2 credentials to a third party to grant read-only access to Cellar-1 buckets.

Read-only policy example

This policy example grants read-only access to a bucket for another user, using the preceding procedure.

{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": [
          "s3:GetObject",
          "s3:ListBucket"
        ],
        "Effect": "Allow",
        "Resource": "arn:aws:s3:::<bucket-name>/*",
        "Principal": {"AWS": "arn:aws:iam::cellar_xxx"}

      }
    ]
  }

Replace the <bucket-name> with your bucket name in the policy file.

Set the policy to your bucket using s3cmd:

s3cmd --config=<path/to/s3cfg-file> setpolicy ./policy.json s3://<bucket-name>

💡Download the configuration file from Clever Cloud:

s3cmd setpolicy ./policy.json -c path/to/s3cfg.txt s3://<bucket-name>

CORS Configuration

You can set a CORS configuration on your buckets if you need to share resources on websites that don’t have the same origin as the one you are using.

Each CORS configuration can contain multiple rules, defined in an XML document:

<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>console.clever-cloud.com</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
    <ExposeHeader>ETag</ExposeHeader>
  </CORSRule>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3600</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>

Here this configuration has two CORS rules:

• The first rule allows cross-origin requests from the console.clever-cloud.com origin. Allowed cross-origin request methods are PUT, POST and DELETE. Using AllowedHeaders * allows all headers specified in the preflight OPTIONS request in the Access-Control-Request-Headers header. At the end, the ExposeHeader allows the client to access the ETag header in the response it received.
• The second one allows cross-origin GET requests for all origins. The MaxAgeSeconds directive tells the browser how much time (in seconds) it should cache the response of a preflight OPTIONS request for this particular resource.

s3cmd -c s3cfg -s info s3://your-bucket

s3cmd -c s3cfg -s setcors ./cors.xml s3://your-bucket

s3cmd -c s3cfg -s delcors s3://your-bucket

Static hosting

You can use a bucket to host your static website, this blog post describes how to. Be aware that SPA applications won’t work because Clever Cloud proxy serving the bucket needs to find an HTML file that match the route.

For example if your path is /login you need to have a file login.html because the index.html isn’t the default entrypoint to handle the path.

You may use SSG (Static Site Generated) to dynamically generate your content during your build.

Troubleshooting

[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1125)

<div class="copy-icon group-[.copied]/copybtn:hx-hidden hx-pointer-events-none hx-h-4 hx-w-4"></div>
<div class="success-icon hx-hidden group-[.copied]/copybtn:hx-block hx-pointer-events-none hx-h-4 hx-w-4"></div>