Swift Enhancements

13 Mar 2023

The “Swift Enhancements” R&D project is a collaboration between ICEI partners and IBM to extend an existing open project called SwiftHLM, developed by IBM Labs. The goal of this enhancement is to create a high-latency storage backend (like a tape archive system) that makes it easier for users to perform bulk operations of data tiering within a Swift data ring.

With the addition of SwiftHLM to OpenStack Swift, we allow users to control and query the state (on disk or HLM) of Swift object data, including the efficient prefetch of bulk objects from HLM to disk.

SwiftHLM has many backend connectors, and it can work with several high-latency backends that are POSIX/DMAPI compatible.

Tiered data flow that uses SwiftHLM in an OpenStack Swift environment.

SwiftHLM consists of:

  • SwiftHLM Middleware (Proxy nodes)

    • Proxy middleware exposing the enhanced API

  • SwiftHLM Dispatcher (Swift node)

    • Background daemon creates a list of objects, identifies the Storage Node for each object, and dispatches asynchronously to the appropriate Swift Storage Node

  • SwiftHLM Handler (Storage Nodes)

    • Provides/invokes generic interface towards SwiftHLM backend storage

    • Maps objects to files and submits the mapped list to the backend (via Connector)


SwiftHLM requires a backend-specific Connector module (the Connector is not part of the SwiftHLM packaging) that is supplied by the vendor of the backend software/hardware (e.g. IBM Spectrum Archive EE, IBM Spectrum Protect).

High-level architecture for high-latency, low-cost media storage solution


Overall SwiftHLM will enable user driven storage tiering that will augment cloud object storage with a low-cost, cold storage tier (Tape, optical), helping users and Data Centers to reduce the cost of long-term object storage (tape up to 6x cheaper than disk).


Copyright 2024 © All Rights Reserved - Legal Notice
Follow us on Twitter
Follow us on Linkedin
Fenix has received funding from the European Union's Horizon 2020 research and innovation programme through the ICEI project under the grant agreement No. 800858.