Orchard CMS: Exporting and importing linked content items

I am working on a software based on Orchard CMS. My module defines multiple content item types which reference each other:

content items

A DataSource defines the location of data. A DataQuery defines a query taking the referenced data as input and returning a collection of objects with properties. A DataGrid defines how the properties of the objects returned by the DataQuery are displayed in an HTML table.

Now let’s have a look at how the references between content items are handled. As an example, we’ll see how the DataQueries are referencing the DataSources.

First, the ContentPartRecord and ContentPart definition of DataQuery looks like this:

public class DataQueryPartRecord : ContentPartRecord
{
	public virtual ContentItemRecord DataSource { get; set; }
	...
}

public class DataQueryPart : ContentPart<DataQueryPartRecord>
{
	private readonly LazyField<IContent> _dataSource = new LazyField<IContent>();

	public LazyField<IContent> DataSourceField
	{
		get { return _dataSource; }
	}

	public IContent DataSource
	{
		get { return _dataSource.Value; }
		set { _dataSource.Value = value; }
	}

	...
}

Of course my parts have other fields but they are not relevant here.

The handler implements the lazy loading and record updating:

public class DataQueryHandler : ContentHandler
{
	private readonly IContentManager _contentManager;

	public DataQueryHandler(IRepository<DataQueryPartRecord> repository, IContentManager contentManager)
	{
		Filters.Add(StorageFilter.For(repository));
		_contentManager = contentManager;

		OnInitializing<DataQueryPart>(PropertySetHandlers);
		OnLoaded<DataQueryPart>(LazyLoadHandlers);
	}

	private void LazyLoadHandlers(LoadContentContext context, DataQueryPart part)
	{
		// add handlers that will load content just-in-time
		part.DataSourceField.Loader(
			() =>
				part.Record.DataSource == null
					? null
					: _contentManager.Get(part.Record.DataSource.Id, VersionOptions.AllVersions));
	}

	private static void PropertySetHandlers(InitializingContentContext context, DataQueryPart part)
	{
		// add handlers that will update records when part properties are set
		part.DataSourceField.Setter(query =>
		{
			part.Record.DataSource = query == null ? null : query.ContentItem.Record;
			return query;
		});

		// Force call to setter if we had already set a value
		if (part.DataSourceField.Value != null)
			part.DataSourceField.Value = part.DataSourceField.Value;
	}
}

The solution to export and import content items referencing each other would have been easier to describe without the lazy loading but I do have it in my code and I didn’t have the time and motivation to refactor it just for the purpose of this post. But this is not the important part so do not let the lazy loading confuse you.

On the database side, the Migrations.cs file defines the following:

// Creating table DataQueryPartRecord
SchemaBuilder.CreateTable("DataQueryPartRecord",
	table =>
		table.ContentPartRecord()
			.Column<int>("DataSource_Id");

ContentDefinitionManager.AlterPartDefinition("DataQueryPart",
	part =>
		part.Attachable()
			.WithDescription(
				"A very nice description..."));

ContentDefinitionManager.AlterTypeDefinition("DataQuery",
	cfg =>
		cfg.WithPart("CommonPart")
			.WithPart("RoutePart")
			.WithPart("DataQueryPart")
			.WithPart("LocalizationPart")
			.WithPart("IdentityPart")
			.Creatable()
			.Draftable()
			.Indexed());

Of course I actually have more columns in my table but they are not relevant here. What’s important is that the table for my ContentPartRecord contains an int column called “DataSource_Id” and referencing the ID of the DataSource record and that my content type has an IdentityPart. The IdentityPart is required anyway in order to be able to properly import the exported data and especially make sure that you update the appropriate content item if it already exists when you import the data.

Now as described in a previous post you need to implement the Importing and Exporting methods in your driver. My first implementation was exporting the referenced DataSource ID as part of the data for the DataQuery:

context.Element(part.PartDefinition.Name).SetAttributeValue("DataSource", part.DataSource.Id);

And on import, I was fetching the DataSource from the ID:

int dataSourceId = int.Parse(context.Attribute(part.PartDefinition.Name, "DataSource"));
part.DataSource = _contentManager.Get(dataSourceId, VersionOptions.AllVersions);

Now the problem is that when I create new content items they get a incrementing ID and when I delete some of them, there is gap in the IDs. Also on the other server where I will import the data, the ID of a content item on the original server might be already in use for a completely different content item. So the content item ID will always be reassigned. This means that when you import the DataSource it might get a different ID. When the DataQuery is imported, the exported ID might now point nowhere or to a completely different content item.

So what we need is some kind of ID which is maintained during export and import. Well, that’s exactly what the IdentityPart is for. This is what Orchard uses to identify whether a content item already exists and needs to be inserted or updated. The IdentityPart contains a unique Identifier which can be exported and then used during import to look up the appropriate content item:

protected override void Importing(DataQueryPart part, ImportContentContext context)
{
	...
	
	string dataSourceIdentifier = context.Attribute(part.PartDefinition.Name, "DataSourceIdentifier");
	part.DataSource = _contentManager
                .Query<IdentityPart, IdentityPartRecord>(VersionOptions.Latest)
                .Where(p => p.Identifier == dataSourceIdentifier)
                .List<ContentItem>().FirstOrDefault();
}

protected override void Exporting(DataQueryPart part, ExportContentContext context)
{
	...
	
	context.Element(part.PartDefinition.Name).SetAttributeValue("DataSourceIdentifier", part.DataSource.As<IdentityPart>().Identifier);
}

So now we got rid of the problem with changing IDs. But there is still a remaining issue. The export file contains all exported content item sorted by content type alphabetically. So the DataQueries come before the DataSources. In case the DataQuery being imported references a DataSource which does already exist, resolving the Identifier to a content item will work. But otherwise it will fail and return null.

Luckily, there is an easy workaround: when the resolved content item is null (i.e. it’s not been imported yet) we can create a dummy content item of the appropriate type, with the right Identifier and no other data. This will solve the referencing problem. And once the DataSources are processed, Orchard will update the dummy content item to contain the actual data for this content item. This only requires a small change in the Importing method:

protected override void Importing(DataQueryPart part, ImportContentContext context)
{
	...
	
	string dataSourceIdentifier = context.Attribute(part.PartDefinition.Name, "DataSourceIdentifier");
	part.DataSource = _contentManager
                .Query<IdentityPart, IdentityPartRecord>(VersionOptions.Latest)
                .Where(p => p.Identifier == dataSourceIdentifier)
                .List<ContentItem>().FirstOrDefault();
	if (part.DataSource == null)
	{
		var item = _contentManager.New("DataSource");
		var dataSourcePart = item.As<IdentityPart>();
		dataSourcePart.Identifier = dataSourceIdentifier;
		_contentManager.Create(item);
		part.DataSource = item;
	}
}

Using this we can import content items in any order and have placeholders created if the items are imported out of order and also make sure that references are kept through the export/import process.

This all may seem quite complex but once you’ve implemented it for one content type e.g. DataQuery, all you need is some copy&paste and search&replace for the other content types e.g. DataGrid.

Also, if for some reason you need to store a reference to a content item in a text form, it’s important to make sure that all referenced content items have an IdentityPart and that you reference the Identifier of this part instead of the ID of the content item. Getting the Identifier of a content item which has an IdentityPart is very easy:

var item = ... // some content item
var identityPart = item.As<IdentityPart>();
var identifier = identityPart.Identifier;

 

Leave a Reply

Your email address will not be published. Required fields are marked *