current situation:
* codfw: swift 1.13.1 (icehouse) on trusty
* eqiad: swift 1.13.1 (icehouse) on precise (save for 6 machines on trusty)
* esams: swift 1.13.1 (icehouse) on precise (cluster not used)
swift changelog: https://github.com/openstack/swift/blob/master/CHANGELOG
jessie ships with swift 2.2 (juno) though the latest upstream version is 2.5 (liberty)
```lines=10,name=changelog diff 2.2.0 -> 2.5
swift (2.5.0, OpenStack Liberty)
* Added the ability to specify ranges for Static Large Object (SLO)
segments.
* Replicator configs now support an "rsync_module" value to allow
for per-device rsync modules. This setting gives operators the
ability to fine-tune replication traffic in a Swift cluster and
isolate replication disk IO to a particular device. Please see
the docs and sample config files for more information and
examples.
* Significant work has gone in to testing, fixing, and validating
Swift's erasure code support at different scales.
* Swift now emits StatsD metrics on a per-policy basis.
* Fixed an issue with Keystone integration where a COPY request to a
service account may have succeeded even if a service token was not
included in the request.
* Ring validation now warns if a placement partition gets assigned to the
same device multiple times. This happens when devices in the ring are
unbalanced (e.g. two servers where one server has significantly more
available capacity).
* Various other minor bug fixes and improvements.
swift (2.4.0)
* Dependency changes
- Added six requirement. This is part of an ongoing effort to add
support for Python 3.
- Dropped support for Python 2.6.
* Config changes
- Recent versions of Python restrict the number of headers allowed in a
request to 100. This number may be too low for custom middleware. The
new "extra_header_count" config value in swift.conf can be used to
increase the number of headers allowed.
- Renamed "run_pause" setting to "interval" (current configs with
run_pause still work). Future versions of Swift may remove the
run_pause setting.
* Versioned writes middleware
The versioned writes feature has been refactored and reimplemented as
middleware. You should explicitly add the versioned_writes middleware to
your proxy pipeline, but do not remove or disable the existing container
server config setting ("allow_versions"), if it is currently enabled.
The existing container server config setting enables existing
containers to continue being versioned. Please see
http://swift.openstack.org/middleware.html#how-to-enable-object-versioning-in-a-swift-cluster
for further upgrade notes.
* Allow 1+ object-servers-per-disk deployment
Enabled by a new > 0 integer config value, "servers_per_port" in the
[DEFAULT] config section for object-server and/or replication server
configs. The setting's integer value determines how many different
object-server workers handle requests for any single unique local port
in the ring. In this mode, the parent swift-object-server process
continues to run as the original user (i.e. root if low-port binding
is required), binds to all ports as defined in the ring, and forks off
the specified number of workers per listen socket. The child, per-port
servers drop privileges and behave pretty much how object-server workers
always have, except that because the ring has unique ports per disk, the
object-servers will only be handling requests for a single disk. The
parent process detects dead servers and restarts them (with the correct
listen socket), starts missing servers when an updated ring file is
found with a device on the server with a new port, and kills extraneous
servers when their port is found to no longer be in the ring. The ring
files are stat'ed at most every "ring_check_interval" seconds, as
configured in the object-server config (same default of 15s).
In testing, this deployment configuration (with a value of 3) lowers
request latency, improves requests per second, and isolates slow disk
IO as compared to the existing "workers" setting. To use this, each
device must be added to the ring using a different port.
* Do container listing updates in another (green)thread
The object server has learned the "container_update_timeout" setting
(with a default of 1 second). This value is the number of seconds that
the object server will wait for the container server to update the
listing before returning the status of the object PUT operation.
Previously, the object server would wait up to 3 seconds for the
container server response. The new behavior dramatically lowers object
PUT latency when container servers in the cluster are busy (e.g. when
the container is very large). Setting the value too low may result in a
client PUT'ing an object and not being able to immediately find it in
listings. Setting it too high will increase latency for clients when
container servers are busy.
* TempURL fixes (closes CVE-2015-5223)
Do not allow PUT tempurls to create pointers to other data.
Specifically, disallow the creation of DLO object manifests via a PUT
tempurl. This prevents discoverability attacks which can use any PUT
tempurl to probe for private data by creating a DLO object manifest and
then using the PUT tempurl to head the object.
* Ring changes
- Partition placement no longer uses the port number to place
partitions. This improves dispersion in small clusters running one
object server per drive, and it does not affect dispersion in
clusters running one object server per server.
- Added ring-builder-analyzer tool to more easily test and analyze a
series of ring management operations.
- Stop moving partitions unnecessarily when overload is on.
* Significant improvements and bug fixes have been made to erasure code
support. This feature is suitable for beta testing, but it is not yet
ready for broad production usage.
* Bulk upload now treats user xattrs on files in the given archive as
object metadata on the resulting created objects.
* Emit warning log in object replicator if "handoffs_first" or
"handoff_delete" is set.
* Enable object replicator's failure count in swift-recon.
* Added storage policy support to dispersion tools.
* Support keystone v3 domains in swift-dispersion.
* Added domain_remap information to the /info endpoint.
* Added support for a "default_reseller_prefix" in domain_remap
middleware config.
* Allow SLO PUTs to forgo per-segment integrity checks. Previously, each
segment referenced in the manifest also needed the correct etag and
bytes setting. These fields now allow the "null" value to skip those
particular checks on the given segment.
* Allow rsync to use compression via a "rsync_compress" config. If set to
true, compression is only enabled for an rsync to a device in a
different region. In some cases, this can speed up cross-region
replication data transfer.
* Added time synchronization check in swift-recon (the --time option).
* The account reaper now runs faster on large accounts.
* Various other minor bug fixes and improvements.
swift (2.3.0, OpenStack Kilo)
* Erasure Code support (beta)
Swift now supports an erasure-code (EC) storage policy type. This allows
deployers to achieve very high durability with less raw capacity as used
in replicated storage. However, EC requires more CPU and network
resources, so it is not good for every use case. EC is great for storing
large, infrequently accessed data in a single region.
Swift's implementation of erasure codes is meant to be transparent to
end users. There is no API difference between replicated storage and
EC storage.
To support erasure codes, Swift now depends on PyECLib and
liberasurecode. liberasurecode is a pluggable library that allows for
the actual EC algorithm to be implemented in a library of your choosing.
As a beta release, EC support is nearly fully feature complete, but it
is lacking support for some features (like multi-range reads) and has
not had a full performance characterization. This feature relies on
ssync for durability. Deployers are urged to do extensive testing and
not deploy production data using an erasure code storage policy.
Full docs are at http://swift.openstack.org/overview_erasure_code.html
* Add support for container TempURL Keys.
* Make more memcache options configurable. connection_timeout,
pool_timeout, tries, and io_timeout are all now configurable.
* Swift now supports composite tokens. This allows another service to
act on behalf of a user, but only with that user's consent.
See http://swift.openstack.org/overview_auth.html for more details.
* Multi-region replication was improved. When replicating data to a
different region, only one replica will be pushed per replication
cycle. This gives the remote region a chance to replicate the data
locally instead of pushing more data over the inter-region network.
* Internal requests from the ratelimit middleware now properly log a
swift_source. See http://swift.openstack.org/logs.html for details.
* Improved storage policy support for quarantine stats in swift-recon.
* The proxy log line now includes the request's storage policy index.
* Ring checker has been added to swift-recon to validate if rings are
built correctly. As part of this feature, storage servers have learned
the OPTIONS verb.
* Add support of x-remove- headers for container-sync.
* Rings now support hostnames instead of just IP addresses.
* Swift now enforces that the API version on a request is valid. Valid
versions are configured via the valid_api_versions setting in swift.conf
* Various other minor bug fixes and improvements.
swift (2.2.2)
* Data placement changes
This release has several major changes to data placement in Swift in
order to better handle different deployment patterns. First, with an
unbalance-able ring, less partitions will move if the movement doesn't
result in any better dispersion across failure domains. Also, empty
(partition weight of zero) devices will no longer keep partitions after
rebalancing when there is an unbalance-able ring.
Second, the notion of "overload" has been added to Swift's rings. This
allows devices to take some extra partitions (more than would normally
be allowed by the device weight) so that smaller and unbalanced clusters
will have less data movement between servers, zones, or regions if there
is a failure in the cluster.
Finally, rings have a new metric called "dispersion". This is the
percentage of partitions in the ring that have too many replicas in a
particular failure domain. For example, if you have three servers in a
cluster but two replicas for a partition get placed onto the same
server, that partition will count towards the dispersion metric. A
lower value is better, and the value can be used to find the proper
value for "overload".
The overload and dispersion metrics have been exposed in the
swift-ring-build CLI tools.
See http://docs.openstack.org/developer/swift/overview_ring.html
for more info on how data placement works now.
* Improve replication of large out-of-sync, out-of-date containers.
* Added console logging to swift-drive-audit with a new log_to_console
config option (default False).
* Optimize replication when a device and/or partition is specified.
* Fix dynamic large object manifests getting versioned. This was not
intended and did not work. Now it is properly prevented.
* Fix the GET's response code when there is a missing segment in a
large object manifest.
* Change black/white listing in ratelimit middleware to use sysmeta.
Instead of using the config option, operators can set
"X-Account-Sysmeta-Global-Write-Ratelimit: WHITELIST" or
"X-Account-Sysmeta-Global-Write-Ratelimit: BLACKLIST" on an account to
whitelist or blacklist it for ratelimiting. Note: the existing
config options continue to work.
* Use TCP_NODELAY on outgoing connections.
* Improve object-replicator startup time.
* Implement OPTIONS verb for storage nodes.
* Various other minor bug fixes and improvements.
swift (2.2.1)
* Swift now rejects object names with Unicode surrogates.
* Return 403 (instead of 413) on unauthorized upload when over account
quota.
* Fix a rare condition when a rebalance could cause swift-ring-builder
to crash. This would only happen on old ring files when "rebalance"
was the first command run.
* Storage node error limits now survive a ring reload.
* Speed up reading and writing xattrs for object metadata by using larger
xattr value sizes. The change is moving from 254 byte values to 64KiB
values. There is no migration issue with this.
* Deleted containers beyond the reclaim age are now properly reclaimed.
* Full Simplified Chinese translation (zh_CN locale) for errors and logs.
* Container quota is now properly enforced during cross-account COPY.
* ssync replication now properly uses the configured replication_ip.
* Fixed issue were ssync did not replicate custom object headers.
* swift-drive-audit now has the 'unmount_failed_device' config option
(default to True) that controls if the process will unmount failed
drives or not.
* swift-drive-audit will now dump drive error rates to a recon file.
The file location is controlled by the 'recon_cache_path' config value
and it includes each drive and its associated number of errors.
* When a filesystem does't support xattr, the object server now returns
a 507 Insufficient Storage error to the proxy server.
* Clean up empty account and container partitions directories if they
are empty. This keeps the system healthy and prevents a large number
of empty directories from slowing down the replication process.
* Show the sum of every policy's amount of async pendings in swift-recon.
* Various other minor bug fixes and improvements.
```
there are jessie backports provided at http://liberty-jessie.pkgs.mirantis.com/ we could use/import. Dependencies to backport are relatively self contained: `python-eclib`, `liberasurecode`, `python-eventlet`