Skip to content

Concepts

Overview

Object Storage provides a highly fault-tolerant and scalable solution for storing and retrieving arbitrary unstructured data objects via an HTTP-based API. Built on top of Ceph, an open-source distributed storage system, our service ensures data redundancy and high availability through replication across regions. It offers S3 compatibility, making it ideal for a wide range of applications.

Object Storage User Required

To use our S3-compatible object storage API, first create an Object Storage user.

Key Features

  • S3 Compatibility: Integrate seamlessly with existing tools and applications that support the S3 API.
  • Scalability: Scale your storage without capacity limits.
  • High Durability and Availability: Object Storage replicates data across redundant nodes to prevent loss and ensure constant availability.
  • Secure Access: Manage access with fine-grained permissions and secure authentication mechanisms.

Note on Replication Consistency

S3 replication runs asynchronously in the background. Updates made in the master zone are not synchronously visible in the replicated zone. Do not assume real-time data consistency across zones. If you need immediate access to the latest data, direct all write and read operations to the master zone.

Use Cases

  • Backup and Archiving: Securely store backups and archives with high durability.
  • Content Storage and Delivery: Host images, videos, and other static content for web and mobile applications.
  • Large-Scale Data Analytics: Store large datasets for analytics and machine learning workloads.
  • Disaster Recovery: Build robust recovery strategies by keeping data offsite.

Buckets and Objects

Buckets

Buckets are the logical units used to store objects. Each bucket name must be unique across all customers in an Object Storage region.

Naming Guidelines:

  • Avoid common and short words
  • Combine rare words
  • Prefix with your company or brand name
  • Add random numbers or hashes (for example, syseleven-bucketname-example-7hg3x)

When creating buckets programmatically, expect to retry with different names or take manual action if conflicts occur.

Objects

Objects are the individual pieces of data stored in buckets, identified by unique keys. Object Storage functions as a key/value store.

Naming Conventions

We discourage special characters in bucket or object names, dots (.) and slashes (/) included. Placing them at the start or end causes problems. These characters can confuse server and client software, potentially making buckets or objects inaccessible or unmaintainable. If you encounter naming-related issues, contact Support (support@syseleven.de).

API Compatibility with Amazon S3

Object Storage supports S3 operations, with some limitations compared to Amazon S3. The following table outlines the supported features:

Feature Supported
bucket HEAD/GET/PUT/DELETE Yes
object HEAD/GET/PUT/POST/DELETE Yes
multipart upload Yes
bucket listing Yes
input compression Yes
request signature authentication Yes
predefined ACL groups Yes
bucket/object ACL Partial
bucket policies Yes
object versions Yes
object expiration Yes
SSE-C encryption Yes
setting CORS headers Yes
website hosting configuration No
bucket/IAM user ACP No
regions and storage classes No
access logging No

Feature Parity with Amazon S3

Our S3-compatible storage may not offer the complete feature set of Amazon S3. Some API calls may fail with a 501 Not Implemented response. If you are unsure about compatibility, please contact Support.

Region Configuration

Due to Ceph S3 API incompatibility, the valid value for region is sys11 or us-east-1. Use region = "sys11" in the master zone (dus2). When creating buckets in the non-master zone (ham1), use region = "us-east-1" to disable the LocationConstraint, because sys11 is not valid outside the master zone. You can also use region = "us-east-1" everywhere, including master and non-master regions.

Encryption at rest

By default, Object Storage encrypts all data at rest. The system encrypts data before writing it to physical devices. A lost or damaged storage device cannot expose your data to unauthorized access.

SSE-C encryption

Users can also encrypt data with their own keys using the SSE-C mechanism, which is compatible with the Amazon SSE-C specification.

In this mode, the user passes an encryption key with each request to write and read encrypted data.

The user manages encryption keys and tracks which key encrypts each object, because the server never stores keys.

SSE-C keys must be 256 bits long and base64 encoded.

Getting Started

To begin using Object Storage, create an Object Storage user and get the necessary credentials.

Create an Object Storage User

  1. Access the Dashboard: Log in to your Dashboard.
  2. Navigate to Object Storage Users: Go to IAM > S3 Users.
  3. Create User: Click on Create S3 User.
  4. Fill Details: - Name: Provide a unique name for the Object Storage user. - Description: Optionally, add a description.
  5. Save Credentials: After creation, securely store the Access Key and Secret Key.

For detailed instructions, see our Object Storage User Management Documentation.

Accessing the Object Storage API

With your Object Storage credentials, you can interact with Object Storage using any S3-compatible client or library.

Endpoints:

  • DUS2 (Düsseldorf - Master zone): https://objectstorage-replicated.dus2.cloud.syseleven.net
  • HAM1 (Hamburg - Replicated zone): https://objectstorage-replicated.ham1.cloud.syseleven.net

Select the endpoint closest to your application or users for optimal performance.

Best Practices

  • Security:
    • Keep your Access Key and Secret Key confidential.
    • Rotate credentials periodically.
  • Performance:
    • Use multipart uploads for files larger than 100 MB.
    • Parallelize uploads and downloads where possible.
  • Data Management:
    • Enable versioning to protect against accidental deletions or overwrites.
    • Set up lifecycle policies to manage object retention and transitions.
  • Monitoring:
    • Set up logging and monitoring to track usage and access patterns.
    • Use tags to organize and identify resources.