Skip to content

SysEleven Object Storage

Overview

SysEleven Object Storage provides a highly fault-tolerant and scalable solution for storing and retrieving arbitrary unstructured data objects via an HTTP-based API. Built on top of Ceph, an open-source distributed storage system, our service ensures data redundancy and high availability through replication across regions. It offers S3 compatibility, making it ideal for a wide range of applications.

Object Storage User Required

To use our S3-compatible object storage API, please first create an Object Storage user.

Key Features

  • S3 Compatibility: Integrate seamlessly with existing tools and applications that support the S3 API.
  • Scalability: Effortlessly scale your storage needs without worrying about capacity limits.
  • High Durability and Availability: Data is replicated across multiple nodes to prevent loss and ensure constant availability.
  • Secure Access: Manage access with fine-grained permissions and secure authentication mechanisms.

Note on Replication Consistency

S3 replication operates asynchronously and is processed in the background. As a result, updates made in the master zone are not immediately reflected in the replicated zone. Operations should not assume data consistency across zones in real time. If immediate access to the latest data is required, all write and read operations should be directed to the master zone.

Use Cases

  • Backup and Archiving: Securely store backups and archives with high durability.
  • Content Storage and Delivery: Host images, videos, and other static content for web and mobile applications.
  • Big Data Analytics: Store large datasets for analytics and machine learning workloads.
  • Disaster Recovery: Implement robust recovery strategies by keeping data offsite.

Buckets and Objects

Buckets

Buckets are the logical units used to store objects. Each bucket name must be unique across all customers in a SysEleven Object Storage region.

Naming Guidelines:

  • Avoid common and short words
  • Combine multiple rare words
  • Prefix with your company or brand name
  • Add random numbers or hashes (e.g., syseleven-bucketname-example-7hg3x)

When creating buckets programmatically, be prepared to retry with different names or resort to manual action if conflicts occur.

Objects

Objects are the individual pieces of data stored in buckets, identified by unique keys. SysEleven Object Storage functions as a key/value store.

Naming Conventions

We discourage the use of special characters, especially dots (.) and slashes (/), in bucket or object names, particularly at the start and end. This can lead to confusion in server and client software, potentially making buckets or objects inaccessible or unmaintainable. If you encounter issues related to naming, please contact our Support (support@syseleven.de).

API Compatibility with Amazon S3

SysEleven Object Storage supports various S3 operations, with some limitations compared to Amazon S3. The following table outlines the supported features:

Feature Supported
bucket HEAD/GET/PUT/DELETE Yes
object HEAD/GET/PUT/POST/DELETE Yes
multipart upload Yes
bucket listing Yes
input compression Yes
request signature authentication Yes
predefined ACL groups Yes
bucket/object ACL Partial
bucket policies Yes
object versions Yes
object expiration Yes
SSE-C encryption Yes
setting CORS headers Yes
website hosting configuration No
bucket/IAM user ACP No
regions and storage classes No
access logging No

Note

Our S3-compatible storage may not offer the complete feature set of Amazon S3. Some API calls may fail with a 501 Not Implemented response. If you're unsure about compatibility, please contact our Support.

Note

Due to Ceph S3 API incompatibility, the valid value for region can be sys11 or us-east-1, region = "sys11" must be used only in the master zone (in our case dus2). When creating buckets in the non-master zone (ham1), use region = "us-east-1" to disable the LocationConstraint, as sys11 is not valid outside the master zone. You can also use region = "us-east-1" everywhere including master and non-master regions.

Encryption at rest

By default all data stored in the S3 is encrypted at rest, which means that data is encrypted before writing it to a physical device. When storage device is lost or damaged your data is protected against unauthorized access.

SSE-C encryption

Users may additionally encrypt data with their own encryption keys using SSE-C mechanism, which is compatible with the Amazon SSE-C specification.

In this mode, the user passes an encryption key along with each request to write and read encrypted data.

The user is responsible for managing encryption keys and information which key was used to encrypt each object, because keys are not stored on the server side in any way.

SSE-C keys must be 256-bit long and base64 encoded.

Getting Started

To begin using SysEleven Object Storage, you need to create an Object Storage user and obtain the necessary credentials.

Create an Object Storage User

  1. Access the Dashboard: Log in to your SysEleven Dashboard.
  2. Navigate to Object Storage Users: Go to IAM > S3 Users.
  3. Create User: Click on Create S3 User.
  4. Fill Details:
    • Name: Provide a unique name for the Object Storage user.
    • Description: Optionally, add a description.
  5. Save Credentials: After creation, securely store the Access Key and Secret Key.

For detailed instructions, refer to our Object Storage User Management Documentation.

Accessing the Object Storage API

With your Object Storage credentials, you can interact with SysEleven Object Storage using any S3-compatible client or library.

Endpoints:

  • DUS2 (Düsseldorf - Master zone): https://objectstorage-replicated.dus2.cloud.syseleven.net
  • HAM1 (Hamburg - Replicated zone): https://objectstorage-replicated.ham1.cloud.syseleven.net

Select the endpoint closest to your application or users for optimal performance.

Best Practices

  • Security:
    • Keep your Access Key and Secret Key confidential.
    • Rotate credentials periodically.
  • Performance:
    • Use multipart uploads for files larger than 100 MB.
    • Parallelize uploads and downloads where possible.
  • Data Management:
    • Enable versioning to protect against accidental deletions or overwrites.
    • Implement lifecycle policies to manage object expiration and transitions.
  • Monitoring:
    • Set up logging and monitoring to track usage and access patterns.
    • Use tags to organize and identify resources.