Skip to content

Firecracker fails with stargz snapshotter and multi-platform image #1137

Description

@leokondrashov

Describe the bug
Firecracker-containerd with stargz snapshotter fails to use multi-platform images in OCI format.

To Reproduce
Run vHive with stargz snapshotter and deploy an image that has application/vnd.oci.image.index.v1+json type (can check with skopeo inspect or ctr-remote image list).

Expected behavior
Runs without failures.

Observerd behaviour
The image fails to be pulled inside the VM, with log messages indicating a missing mount.

Failed to start VM: Failed to get/pull image: failed to extract layer sha256:2803a94be65d07d4aad0fbd0998486d1e17908a2af321be5c73344262609d987: failed to mount /var/lib/firecracker-containerd/containerd/tmpmounts/containerd-mount1115787544: no such device: unknown

Workaround
Existing workaround is to convert images to a single platform, by push --platform linux/amd64 in ctr-remote or docker

Notes
Demux snapshotter complains about a missing namespace in the context:

ERRO[0013] Function called without namespaced context    error="namespace is required: failed precondition" function=Walk
DEBU[0013] no namespace found, proxying walk function to all cached snapshotters  function=Walk
ERRO[0013] Function called without namespaced context    error="namespace is required: failed precondition" function=Remove
ERRO[0013] Function called without namespaced context    error="namespace is required: failed precondition" function=Remove
ERRO[0013] Function called without namespaced context    error="namespace is required: failed precondition" function=Cleanup

This doesn't happen with other types of images (application/vnd.oci.image.manifest.v1+json or application/vnd.docker.distribution.manifest.v2+json).

Another clue might be that the firecracker-containerd log has lines about cancelled context, also only for the same images, and since the namespace is propagated through the context, that might be connected.

level=error msg="error while forwarding events from VM agent" error="rpc error: code = Canceled desc = context canceled" runtime=aws.firecracker vmID=auth-go-27496

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions