Drop the old tarsum computation for Docker <0.10

This landed in 82021b4 (Merging bypass-tarsum, 2014-04-07, #300), but
those Dockers are now ancient history.

I'm against this sort of user-agent-dependent behavior in general.  If
we want to version the API, it makes sense to have version checks in
one place (e.g. in the URL, or in an explicit Docker-Version header,
or in the PUT payload, ...). I'm happy with either the URL approach
(always explicit) or the Docker-Version approach (defaulting to the
current version). If we make some small change to the API (such as not
storing layer tarsums), and want to add per-feature headers or
registry-side options to enable/disable this behaviour, that's
fine. If we can auto-detect new features (e.g. if
Docker-Checksum-Payload exists, it must be using the new SHA-256
semantics), that's fine too. But regardless of how we version the API,
conflating the API version with the user-agent version just seems
confusing.

Olivier wants us to warn older clients instead of just failing when
their tarsum doesn't match our checksum [1], so we still need to use
their user-agent string to decide which clients get warned.  I've
softened the old DockerVersion into a more relaxed
docker_client_version, which no longer rejects non-Docker clients [2].
If we can't version the API, we can at least use a blacklist to deny
old implementations instead of a whitelist that requires other clients
to fake the Docker-daemon User-Agent string.

For the 'PUT /v1/images/<image_id>/checksum' endpoint, we don't need
to rely on the user-agent, because the old X-Docker-Checksum was
replaced by the new X-Docker-Checksum-Payload.  If the client isn't
setting X-Docker-Checksum-Payload, just say that in the error message
and hint that it might be due to an outdated client.

I'd like to remove all the user-agent-dependent behavior, but we're
currently also using it to setup some tag metadata in
docker_registry.tags.put_tag.  I don't touch that here, since I
haven't tracked down the reasoning behind that behavior (although I
suspect it's also a bad idea ;).

I also haven't touched the user-agent bits under tests/, since I
haven't checked to see if they're designed to test this check, or to
test the put_tags check, or just to bypass either of those checks.

[1]: #570 (comment)
[2]: #375

Author: wking

docker_registry/images.py

@@ -194,6 +194,11 @@ def get_image_layer(image_id, headers):
 @app.route('/v1/images/<image_id>/layer', methods=['PUT'])
 @toolkit.requires_auth
 def put_image_layer(image_id):
+    client_version = toolkit.docker_client_version()
+    if client_version and client_version < (0, 10):
+        return toolkit.api_error(
+            'This endpoint does not support Docker daemons older than 0.10',
+            412)
     try:
         json_data = store.get_content(store.image_json_path(image_id))
     except exceptions.FileNotFoundError:
@@ -210,39 +215,11 @@ def put_image_layer(image_id):
     # compute checksums
     csums = []
     sr = toolkit.SocketReader(input_stream)
-    if toolkit.DockerVersion() < '0.10':
-        tmp, store_hndlr = storage.temp_store_handler()
-        sr.add_handler(store_hndlr)
     h, sum_hndlr = checksums.simple_checksum_handler(json_data)
     sr.add_handler(sum_hndlr)
     store.stream_write(layer_path, sr)
     csums.append('sha256:{0}'.format(h.hexdigest()))
 
-    if toolkit.DockerVersion() < '0.10':
-        # NOTE(samalba): After docker 0.10, the tarsum is not used to ensure
-        # the image has been transfered correctly.
-        logger.debug('put_image_layer: Tarsum is enabled')
-        tar = None
-        tarsum = checksums.TarSum(json_data)
-        try:
-            tmp.seek(0)
-            tar = tarfile.open(mode='r|*', fileobj=tmp)
-            tarfilesinfo = layers.TarFilesInfo()
-            for member in tar:
-                tarsum.append(member, tar)
-                tarfilesinfo.append(member)
-            layers.set_image_files_cache(image_id, tarfilesinfo.json())
-        except (IOError, tarfile.TarError) as e:
-            logger.debug('put_image_layer: Error when reading Tar stream '
-                         'tarsum. Disabling TarSum, TarFilesInfo. '
-                         'Error: {0}'.format(e))
-        finally:
-            if tar:
-                tar.close()
-            # All data have been consumed from the tempfile
-            csums.append(tarsum.compute())
-            tmp.close()
-
     # We store the computed checksums for a later check
     save_checksums(image_id, csums)
     return toolkit.response()
@@ -251,10 +228,12 @@ def put_image_layer(image_id):
 @app.route('/v1/images/<image_id>/checksum', methods=['PUT'])
 @toolkit.requires_auth
 def put_image_checksum(image_id):
-    if toolkit.DockerVersion() < '0.10':
-        checksum = flask.request.headers.get('X-Docker-Checksum')
-    else:
-        checksum = flask.request.headers.get('X-Docker-Checksum-Payload')
+    checksum = flask.request.headers.get('X-Docker-Checksum-Payload')
+    if checksum is None:
+        return toolkit.api_error(
+            ('X-Docker-Checksum-Payload not set.  If you are using the Docker '
+             'daemon, you should upgrade to version 0.10 or later'),
+            412)
     if not checksum:
         return toolkit.api_error('Missing Image\'s checksum')
     if not store.exists(store.image_json_path(image_id)):

docker_registry/toolkit.py

@@ -1,7 +1,6 @@
 # -*- coding: utf-8 -*-
 
 import base64
-import distutils.version
 import functools
 import logging
 import os
@@ -26,17 +25,24 @@
 _re_authorization = re.compile(r'(\w+)[:=][\s"]?([^",]+)"?')
 
 
-class DockerVersion(distutils.version.StrictVersion):
+def docker_client_version():
+    """Try and extract the client version from the User-Agent string
 
-    def __init__(self):
-        ua = flask.request.headers.get('user-agent', '')
-        m = _re_docker_version.search(ua)
-        if not m:
-            raise RuntimeError('toolkit.DockerVersion: cannot parse version')
-        version = m.group(1)
-        if '-' in version:
-            version = version.split('-')[0]
-        distutils.version.StrictVersion.__init__(self, version)
+    So we can warn older versions of the Docker engine/daemon about
+    incompatible APIs.  If we can't figure out the version (e.g. the
+    client is not a Docker engine), just return None.
+    """
+    ua = flask.request.headers.get('user-agent', '')
+    m = _re_docker_version.search(ua)
+    if not m:
+        return
+    version = m.group(1)
+    if '-' in version:
+        version = version.split('-')[0]
+    try:
+        return tuple(int(x) for x in version)
+    except ValueError:
+        return
 
 
 class SocketReader(object):