Image data type

The Image data type is a type that has the particularity of not being represented in the same way in the different layers of the framework. On the server side, the Image data type works exactly like the File data type. The difference is on the client side, where the Image data type is loaded directly and displayed in the template.

Entity

How to configure the MIME type?

You have two choices:

If your images are of the same type all the time, there is no need to store this information in the database. In this case you can choose the Not persisted mode and enter the MIME type of the image directly.
If your images are not all the time of the same type, it is necessary to store this information in database. In this case you have to choose the Persisted mode and enter the column name.

Note

It is also possible to use the generic MIME type image/* but not all images will be automatically recognized (example: svg files).

How to read an image?

An Image property can be read entirely or property by property to limit the downloaded volume:

IQueryable<Product> query = _repository.GetQuery();

// Reading the whole object
BinaryFile picture = query.Where(p => p.ID == 1).Select(p => p.Picture).First();

// Reading the MIME type only
string pictureMimeType = query.Where(p => p.ID == 1).Select(p => p.Picture.MimeType).First();

How to write an image?

A BinaryFile is a value object and is immutable. To modify an image, you have to create a new instance and assign it to your entity property:

Product p = await _repository.GetAsync(1);

// Updating the file name by creating a new instance
p.Picture = new BinaryFile("NewName.jpeg", p.Picture.Content, p.Picture.MimeType);

await _unitOfWork.SaveAsync();

Entity view

How does standard reading / writing work?

As said above, in an entity view, an image is represented as a string property. With an entity view named ProductView containing a Picture property, the returned data will look like this:

[
  {
    "id": 32,
    "picture": "productview/32/picture"
  },
  {
    "id": 33,
    "picture": "productview/33/picture"
  }
]

This partial URL must be used by the API client to build an absolute URL (for example https://localhost/neos/MyCluster/webapi/productview/32/picture) which will allow the image to be fetched. If the entity view property is mapped to an entity property, the image GET API is automatically generated and everything should work.

To write an image, two possibilities are available:

Access to the entity from the entity view and update it directly:

Product entity = entityView.GetEntity();
entity.Picture = new BinaryFile("Product1.jpeg", content, "image/jpeg");
await _unitOfWork.SaveAsync();

Store an image in the temporary storage and update the entity view with its Guid:

BinaryFile file = GetFile();
Guid identifier = await temporaryFileStorage.AddAsync(file);
entityView.Picture = identifier.ToString();
await _unitOfWork.SaveAsync();

The use of the temporary storage on the server side is not really advantageous. This mechanism is mainly useful on the client side to dissociate the action of uploading the image from the action of saving it as we will see below.

How to handle an unbound image property?

To manage unbound images, you have to use the flexibility of server methods.

To illustrate how this works, let's take an entity view called ProductView in which we want to expose a Thumbnail property containing a thumbnail of the product.

First, create the unbound Image property named Thumbnail with this getter:

return $"ProductView/{Item.ID}/Thumbnail";

Then create a server method to respond to the URL returned by the property:

Name = GetProductThumbnail
Exposed as API = true
HTTP method = Get
Route = ProductView/{id}/Thumbnail

The return type in the implementation must be a IFileResult:

/// <inheritdoc/>
public IFileResult Execute(int id)
{
  byte[] thumbnail = BuildThumbnail(id);
  return new FileContentResult("image/jpeg", thumbnail);
}

or a Task<IFileResult>:

/// <inheritdoc/>
public async Task<IFileResult> ExecuteAsync(int id)
{
  byte[] thumbnail = await BuildThumbnailAsync(id);
  return new FileContentResult("image/jpeg", thumbnail);
}

You can return a content, a stream or redirect to a URL :

classDiagram
  class IFileResult{
    <<interface>>
    +string ContentType
    +string? FileName
  }

  class IFileContentResult{
    <<interface>>
    +byte[] FileContents
  }

  class IFileStreamResult{
    <<interface>>
    +Stream FileStream
  }

  class IUrlFileResult{
    <<interface>>
    +string Url
  }

  IFileResult <|-- IFileContentResult
  IFileResult <|-- IFileStreamResult
  IFileResult <|-- IUrlFileResult
  IUrlFileResult <|-- UrlFileResult
  IFileContentResult <|-- FileContentResult
  IFileStreamResult <|-- FileStreamResult

This mechanism allows to handle the reading correctly. For writing, you have to write code considering that, if the client has added or modified an image, it has transferred it to the temporary storage and updated the property with its Guid:

public async Task OnSavingAsync(ISavingRuleArguments<IProductView> args)
{
    foreach (IProductView entityView in args.CreatedAndModifiedItems.Where(i => i.Picture != null))
    {
        if (Guid.TryParse(entityView.Picture, out Guid identifier))
        {
            BinaryFile? file = await _temporaryFileStorage.FindAsync(identifier);
            if (file != null)
            {
                ...
            }
        }
    }
}

UI views

How does image reading / writing work?

When an image is received as a partial URL from the server, it is transformed into FileReference. An entity view property called Picture with the value productview/32/picture will produce a FileReference instance looking like :

{
  "Value": "productview/32/picture",
  "Url": "https://localhost/neos/MyCluster/webapi/productview/32/picture",
  "UploadState": "NotStarted",
  "FileName": null
}

The image components only displays an image. To change the image stored in the property it is linked to, the developer must provide an action to show the file selection dialog using the SelectFileAsync or SelectDeferredFileAsync methods of the view model:

FileReference? fileReference1 = await SelectFileAsync("image/jpeg");
if (fileReference1 != null)
{
    Item.Picture1 = fileReference1; // Uploaded immediately
}

DeferredFileReference? fileReference2 = await SelectDeferredFileAsync("image/jpeg");
if (fileReference2 != null)
{
    Item.Picture2 = fileReference2; // Uploaded when calling StartUpload()
    fileReference2.WithCallback(fileReference => {
        // Optional method called when uploading succeeds or fails
    })
    fileReference2.StartUpload();
}

Note

The SelectFileAsync and SelectDeferredFileAsync methods accept several types : It can be a MIME type: SelectFileAsync("image/jpeg", "image/png") It can be a filename extension starting with a period character: SelectFileAsync(".jpeg", ".png")

You can also use another version of the methods to prevent the user from selecting a file that is too large:

int maxAllowedSizeInBytes = 3 * 1024 * 1024; // 3 MB
FileReference? fileReference1 = await SelectFileAsync(maxAllowedSizeInBytes, "image/jpeg");
...

DeferredFileReference? fileReference2 = await SelectDeferredFileAsync(maxAllowedSizeInBytes, "image/jpeg");
...

The selected file size is also available in the Size property of the FileReference instances returned by SelectFileAsync and SelectDeferredFileAsync. In all other cases, Size is null.

At the beginning, the image is only local and Value is empty. As long as the object remains in this state, saving is impossible:

{
  "Value": null,
  "Url": "blob:https://example.org/957b4d22-c5b5-4c5f-b5b5-f7f3b3bf2b05",
  "UploadState": "NotStarted",
  "FileName": "Pict1454878.jpeg"
}

When using SelectFileAsync, the upload to the temporary storage of the server starts as the file is selected. When using SelectDeferredFileAsync, you need to call the StartUpload method:

{
  "Value": null,
  "Url": "blob:https://example.org/957b4d22-c5b5-4c5f-b5b5-f7f3b3bf2b05",
  "UploadState": "InProgress",
  "FileName": "Pict1454878.jpeg"
}

If the upload succeeds, Value is initialized with the image identifier in the temporary storage of the server. This is the value that is passed to the server when saving:

{
  "Value": "F77B6C20-B20D-44BE-983F-7D8BD2CC46FC",
  "Url": "blob:https://example.org/957b4d22-c5b5-4c5f-b5b5-f7f3b3bf2b05",
  "UploadState": "Success",
  "FileName": "Pict1454878.jpeg"
}

If the upload fails, Value remains empty and saving will be impossible. The user can try to reselect the image to restart an upload:

{
  "Value": null,
  "Url": "blob:https://example.org/957b4d22-c5b5-4c5f-b5b5-f7f3b3bf2b05",
  "UploadState": "Failed",
  "FileName": "Pict1454878.jpeg"
}

If a callback method was defined using the WithCallback method on a DeferredFileReference, the callback method is called whether the upload succeeds or fails. The file reference is passed to the callback method which can check the UploadState and act accordingly.

Image loading sequence diagram

The diagram below shows what happens when a screen displays a list of products with an image property.

sequenceDiagram
  participant Client
  participant Server

  Client->>Client: Product list display

  rect rgb(240, 240, 240)
  Client->>Server: Request
  Note right of Client: GET https://localhost/neos/MyCluster/webapi/ProductView/32
  Server->>Client: Response
  Note right of Client: [{ "id": 32, "picture": "productview/32/picture" },<br/>{ "id": 33, "picture": "productview/33/picture" }]
  end

  par
    rect rgb(240, 240, 240)
    Client->>Server: Image download request
    Note right of Client: GET https://localhost/neos/MyCluster/webapi/ProductView/32/picture
    Server->>Client: Image stream Response
    end
  and
    rect rgb(240, 240, 240)
    Client->>Server: Image download request
    Note right of Client: GET https://localhost/neos/MyCluster/webapi/ProductView/33/picture
    Server->>Client: Image stream Response
    end
  end

Image writing sequence diagram

The diagram below shows what happens when the user selects a new image and presses Save :

sequenceDiagram
  participant Client
  participant Server

  Client->>Client: New image selection

  rect rgb(240, 240, 240)
  Client->>Server: Image upload request
  Note right of Client: https://localhost/neos/MyCluster/webapi/$neos/temp-storage/upload
  Server->>Client: Image upload Response
  Note right of Client: F77B6C20-B20D-44BE-983F-7D8BD2CC46FC
  end

  Client->>Client: Save

  rect rgb(240, 240, 240)
  Client->>Server: Request
  Note right of Client: PUT https://localhost/neos/MyCluster/webapi/ProductView/32<br/>{ "id": 32, "picture": "F77B6C20-B20D-44BE-983F-7D8BD2CC46FC" }
  Server->>Client: Response
  end

We can see that the image is first uploaded to the server and that the call to save only contains the identifier of the image in the temporary storage.

Table of Contents

Image data type

Entity

How to configure the MIME type?

Note

How to read an image?

How to write an image?

Entity view

How does standard reading / writing work?

How to handle an unbound image property?

UI views

How does image reading / writing work?

Note

Image loading sequence diagram

Image writing sequence diagram