Adding footnotes to Wagtail

Inline footnote creation within the rich text editor is a requirement that has arisen for me a number of times when building CMS-driven sites.

It came up again on the current project I'm working on whilst still in its first phase in 2018. Due to budget and time constraints, the implemented solution at that time was a halfway house: user-created footnote streamfield instances; instructions to the CMS editors to manually add footnote numbers to the referenced text; using regular "external" links in the Draftail text editor upon saving or previewing the page when the footnotes had been assigned a unique id.

Unsurprisingly, this unrefined implementation was only partly used by the editors, with footnote numbers being added to the rich text, but not being linked to the footnote ids.

Having the ability to revisit the project in a new phase with more time and budget gave me the chance to develop a solution that was more integrated, far less brittle, and delivered a much improved user experience.

There's a ton of code that went into making this working implementation of footnotes in the Wagtail text editor. I'm going to skip code examples as they wouldn't make much sense in isolation (and there's far too much to put up here), but if you want to know more after reading this post give me a shout and I'll be happy to help.

The challenges

Footnotes need to be inline

A footnote link needs to be directly after the referenced text, so we can't use a streamfield block as the Wagtail documentation suggests when deciding whether to extend the rich text editor.

Alas, the Wagtail rich text editor - Draftail (an extension of Draft.js) - has no native footnote or fragment id functionality, so we'll have to roll our own.

Unique fragment identifiers

Whether it's a footnote or not, any fragment identifier on a page needs to be unique. Adding these ids and links inline is a complex challenge, as is visible from the history of this related issue which dates back to 2015.

We also need the ids to stay the same (so that bookmarked or search indexed links always refer to the correct footnote), even if new footnotes are added or previous ones removed.

Extending the Draftail editor

The editor is built to be extendable, however the Draftail implementation requires use of low-level APIs to extend the interface, so in this case it's not a matter of configuring some settings and sitting back with a nice cup of tea. Also, the example documentation is sparse, and doesn't deal with user input in detail.

An easily overlooked consideration is the established and familiar user interface flow and style for adding entities such as links, images, documents, and embeds. Keeping things consistent when performing a new action should be reassuring for the CMS editor and make for a good user experience.

Footnotes may contain links

This functionality existed in the first phase - as well as being required in future editing - so any new implementation needs to include this capability.

Outputting the footnotes in the page template

Once footnotes have been added to one or more rich text editor instances in the page editor, we need to output them in the template, converting the saved data to fragment ids, links, footnote text, and reference links.

Building the new implementation

Proof-of-concept

Starting with the example demo in the Wagtail docs, and some helpful further reading, I first built a proof-of-concept (POC) as suggested by Thibaud Colas, the lead Draftail developer at Torchbox.

In a pretty short amount of time I was able to confirm that I could add a new user interaction to the editor that allowed a user to create, edit, and delete footnotes inline within rich text - the core of the required new functionality.

At this stage I only wanted to deal with the extending the editor, as the template output should be relatively simple.

Links within footnotes

Once I had a working POC, I then tried to extend the new footnote entity with sub-entity links, only to find out this isn't possible after writing up a question on Stack Overflow.

This turned out to be a blessing in disguise, as it simplified the editor interaction and kept the focus on adding footnote content, rather than also editing links using the existing editor flow.

Following this minor setback, I reviewed the requirements and concluded that a simplified textarea field would be sufficient to capture and store the footnote data, as long as footnote links were always of the external type, and did not require labelling (i.e. could be displayed in their raw URL format). Django's magic urlize filter would do the trick here.

If more sophisticated links such as those for relative pages, email addresses, or documents were required, then this solution would need refactoring further.

Getting ready for production

Once I'd worked through the POC (and swerved around link sub-entities), I began making the code production ready, and improving the user interface to make the new interaction feel integrated into the existing Wagtail editor flow.

This involved creating new React elements, and integrating with existing Wagtail admin tooltip and modal workflow elements, reusing these elements where possible.

Finally, the new JS code needed transpiling back to ES5, as IE11 still needs supporting.

Unique id generation

Using unique ids gives us these advantages:

• CMS editors do not need to be concerned with with the "number" of the footnote, as they'll be auto-generated at display time
• Adding, moving, or deleting footnotes doesn't affect the ids, so the references are stable
• Bookmarked or search-indexed links will stay alive as long as the footnote does, even if the order of footnotes on the page is changed

It would've been handy to be able to query other instances of the Draftail editor from the current one when creating new footnote entities, but I couldn't figure out a way to do this. Instead I relied on a six-digit auto-generated id, coupled with a an array of IDs on the window object to check duplicates against. Urgh. Ugly, but it works.

One thing to consider was to ensure that a copy/paste interaction didn't duplicate ids: this is taken care of in the React element which tests how the entity was created and generates new ids if necessary.

The unique id doesn't need to be editable by the CMS user, so this is a hidden field in the footnote form that handles input, and is stored alongside the footnote text data for use in template rendering.

Template output

Once the rich text entity is configured to store and retrieve the new data type, to render a page with footnotes we need to convert the stored data into:

• Numbered links to the footnotes with unique ids (for linking back from the footnote text)
• An ordered list of footnotes with unique ids, and links back to the reference text

This is done in two parts. The first is a custom Django template tag that each footnote-enabled rich text field is passed into. This parses the content looking for the footnote entities, and uses a bit of BeautifulSoup magic to convert the data into a numbered link with a unique return id.

This tag then adds the footnote content and unique id to a list on the page object retrieved from the context.

The second part is a simple Django template partial that loops through the generated list, outputting the footnotes in an ol, and using the urlize filter to convert plain text URLs to interactive links.

Putting it altogether

So what's it like to use?

I spent a decent amount of time ironing out the rough edges so that the new entity feels like a natural part of the editor. Similar to the existing link entity, the CMS editor can insert a new footnote, or select an existing block of text to use as the starting point.

Upon creating a new footnote, the editor is presented with the same modal window flow as other native controls, and once an an entity is created an icon is inserted into the text, with the ability to edit or delete via the Wagtail admin tooltip (which also displays the first 20 characters within the tooltip).

Getting the code into a format that worked with the grain of Wagtail admin meant diving into the source and following some existing patterns: not too hard, but it would be nice if this was available as an abstraction in future versions.

Learnings

• Adding new icons to the Wagtail admin stylesheet is possible, but it isn't trivial, and in the end I found it was better to add the button and entity icon using SVG paths
• The documentation for draft.js and Draftail is limited, so quite a bit of digging was required to discover how the editor works within the Wagtail environment
• It would have been useful to allow editor instances to query each other to verify uniqueness of ids, but the instances appear to be encapsulated
• The inability to wrap entities feels like an unnecessary limitation, but draft.js is still an outstanding editor, and in my experience so far much better than the previous default editor, hallo.js.
• It's important to keep the UI flow as similar as possible to the existing one when creating a new entity

I've wanted to build functionality like this for a while now, so many thanks to the team at Development Initiatives for the chance to build it for a live project.

To see the footnotes in action, visit the page referenced in this post.