Managing legacy decisions

Kenya Law’s legacy case system contains the details of over 150,000 decisions. These must be imported into Tausi without losing information, in particular:

  • primary decision information MUST not be lost, including case content and metadata such as dates, parties, summaries, caseback information, etc.

  • secondary decision information SHOULD not be lost if possible, including user information, emails sent, etc.

Note: in the legacy database, a decision is termed a case.

Legacy dataset

There are two parts to the legacy database, both of which must be imported into Tausi.

  • a mysql database dump

  • a collection of files that are the original Word and PDF documents uploaded into the legacy system

We can divide the legacy dataset broadly into decisions which were selected for reporting, and those which weren’t. Non-reportable decisions have less additional metadata and are simpler to process. These two groups of decisions are imported separately.

Modeling legacy data

Tausi has a module that models the legacy database using Django. This allows us to use normal Django database operations to work with the legacy data.

We use the unmanaged model Django feature to support this.

Preserving historical data

No primary legacy case information can be lost.

In many cases, there is a simple mapping between the fields on a legacy case and a Tausi decision. In other cases, Tausi must transform the legacy data to match the new model.

In particular:

  1. The legacy case ID is stored by Tausi. This allows Tausi to map back to the legacy case when necessary. It also allows the frontend website to understand legacy URLs.

  2. The legacy HTML content is stored by Tausi for historical reference.

Importing legacy decisions

The legacy dataset is imported through the Django admin where an administrator can browse legacy decisions as hosted on the legacy database and easily select the ones to be imported from the provided list.

Import Behavior

When a decision is selected for import, all the available details are mapped onto the new decision model. However, not all details about a legacy decision are available in the legacy data and in that case, Tausi falls back onto some defaults.

The fields that can be missing in the legacy decision data but are required by Tausi are as follows:

  • case_action which defaults to No details

  • case_type which defaults to Miscellaneous Case

  • case_outcome which defaults to No Final Orders

  • filing_year which defaults to delivery_year

This list is not exhaustive and is subject to changes with further development.

Re-importing a decision

A decision will not be imported twice. Use the --reimport flag to tell Tausi to delete the existing decision and re-import it.

Note: all data for the existing decision will be discarded!

The import process

Importing a decision works as follows:

  1. A legacy case is imported as a decision into Tausi using the legacy_import command.

  2. Fields from the legacy case are mapped to the new decision.

  3. The HTML is transformed into Akoma Ntoso markup.

  4. An ingestion workflow is started for the decision, starting at the inventory task.

  5. A user is assigned to the workflow’s inventory task.

  6. The user reviews the imported decision and makes corrections where necessary.

  7. The user submits the decision for review.

  8. The reviewer either makes changes, requests changes from the submitter, or approves the imported decision.

  9. The decision is approved and made available through the public API.

Staged import

Importing will be conducted in stages by filtering on what is being imported. This allows us to incrementally import legacy cases as Tausi supports the required functionality.

  1. Unreported cases without caseback/case histories

  2. Unreported cases with caseback/case histories

  3. Reported cases

Implementation details

  • Where a legacy field needs to be preserved, it should be clearly and consistently named in Tausi’s models, such as legacy_id, legacy_html etc.

Outstanding

  • [ ] what database do we connect to for the legacy data?

  • [ ] where does Tausi fetch the Word and PDF files from?

  • [ ] how much of the processing history can be imported?

  • [ ] can we use Tausi’s audit trail to preserve legacy actions that don’t have an equivalent in Tausi? eg. flagging.

  • [ ] we will need to be able to re-parse HTML into AKN a few times as we iterate on it. This probably needs a management command, and a button in the admin interface.