Java ACDP Tutorial

Maintenance

You don't need to be a certified database administrator to maintain an ACDP database. Maintaining an ACDP database involves:

Making changes to the database settings.
Compacting the database from time to time.
Making structural changes to the database.
Checking and possibly restoring the integrity of the database.

These tasks are covered in the remaining sections of this page:

Changing Settings Compacting Refactoring Checking Integrity

Changing Settings

The layout bundles all information about a database. This includes information from very different categories, such as the names of tables and columns, descriptors of column types, path information of backing data files, dimension sizes, such as the number of bytes for storing row references, and other very technical information.

The information contained in the layout can be read by invoking the info method which is defined on all three levels (database, table, column). For example, to get the type descriptor of the Courses column of the Teacher table call

   try (EduDB db = new EduDB(Paths.get("dbDir/layout"), -1, true, 0)) {
      String desc = TeacherTable.COURSES.info().type().typeDesc());
   }

(The type descriptor describes the type of a table column. It is used to dynamically create the column types at database startup.)

The info method can be used for both a WR and RO database.

If the database is a WR database, information from the layout can be read even when the database is closed. This is done with the Settings class. For example, to get the type descriptor of the Courses column of the Teacher table call

   String desc = new Settings(Paths.get("dbDir/layout")).
                        getTypeDesc("Teacher", "Courses"));

where Teacher is the name of the teacher table and Courses is the name of the Courses column.

Entries in a layout can also be safely changed with the Settings class. For example, the following code changes the name of the Student table from Student to Pupil, the column name Student Number to Pupil Number and adds the version number 1.0 to the database.

   new Settings(Paths.get("dbDir/layout")).
             setTableName("Student", "Pupil").
             setColumnName("Pupil", "Student Number", "Pupil Number").
             setVersion("1.0").save();

It is important to invoke the save method so that the changes are actually saved to the layout file.

Not all entries in the layout can be changed with the Settings class. For example, the type descriptor of the Courses column of the Teacher table cannot be changed with the Settings class, because otherwise the database would no longer work. (The type descriptor in the layout is automatically adjusted when the type of a column is changed using the Refactor class described in Section Refactoring.)

Note: Never change the layout file manually, unless you are a real ACDP expert!

Compacting

ACDP distinguishes between fixed-length (FL) and variable-length (VL) data. This means that the representation of values as a sequence of bytes is either always the same length, regardless of the value, or has a variable length depending on the value. For example, the data of an integer value in the Postal Code column of the Contact table is always four bytes. In contrast, the data of the string values of the Email column have different lengths, depending on the string value.

ACDP stores FL and VL data of a table of a WR database in two separate data files.

When data is inserted, updated and deleted, unused memory blocks, so-called gaps, arise in the data files. For example, inserting 100 rows into a newly created table and deleting 95 results in a FL data file which consists of almost 95% gaps.

Gaps in a data file can be eliminated by a process called compacting, which reduces the size of the file accordingly.

Fortunately, gaps in an FL data file can be filled in by ACDP as new rows are inserted into the table. So, if you reinsert 95 rows into the table mentioned above, all gaps in the FL data file are eliminated. Since unused memory blocks can be reused, compacting an FL data file is only necessary in rare cases.

Unfortunately, this looks different in case of a VL data file. ACDP has very limited ability to fill gaps in VL data files during normal operation. As VL data is inserted, updated and deleted the number of gaps usually increases over time, making it necessary to compact the VL data file of a table from time to time.

The data files can be compacted at any time with the database open using the methods

   compactFL()  // Compacts the FL data file of a table

and

   compactVL()  // Compacts the VL data file of a table

which are declared in the Table interface and in the CustomTable class.

For example, the following program code compacts the VL data file of each table in a database if the size of memory occupied by gaps is larger than the total size of used memory blocks.

   // db supposed to be an instance of CustomDatabase
   // If db is an instance of Database then replace
   // second line of code with
   // for (Table table : db.getTables()) {
   if (db.info().isWritable()) {
      // Since db is writable db is a WR database.
      for (CustomTable table : db.getTables()) {
         final WRStoreInfo info = (WRStoreInfo)
                          table.info().storeInfo();
         if (info.vlUnused() > info.vlUsed()) {
            table.compactVL();
         }
      }
   }

This program code can be executed at the time the database is opened or closed or, if the database is open for a long time, this program code can be executed in a separate thread every few minutes or seconds.

The KeyFigures class prints a report on the most important figures of a database, including the size of the unused memory for each table.

Refactoring

Refactoring a database means changing the structure of the database, for instance, removing a table from the database or modifying the type of a column of a certain table. Refactoring is usually done at a time when the database is already filled with some data. The Refactor class provides methods for refactoring a database.

Let's add to our sample database a new table that stores information about rooms. For each room a number identifying the room and a list of courses held in that room should be stored. The following program code adds the RoomTable under the name Room to our sample database which is assumed to be saved in the dbDir directory:

   Refactor.addTable(Paths.get("dbDir/layout"), "Room",
                  new RoomTable(), Refactor.names().
                  add("Number").add("Courses", "Course"));

Note, that the column types of the Room table are defined in the RoomTable class but the names of the columns (Number and Courses) as well as the name of the referenced Course table ("Course") must be specified using an instance of the Refactor.Names class.

Refactoring changes the database layout: At the very end of the database layout file you will find the table layout for the just added Room table.

As a second example we change the type of the Salary column in the Teacher table from BigInteger to Double and no longer allow null values to be stored in that column:

   Refactor.modifyColumn(Paths.get("dbDir/layout"), null,
          "Teacher", "Salary", CL.ofDouble(Nulls.NO_NULL),
                 big -> ((BigInteger) big).doubleValue());

Remarks:

The expression CL.ofDouble(Nulls.NO_NULL) defines the Salary column to be of type Double, rejecting null values.
Existing BigInteger values are converted to Double values according to the lambda expression big -> ((BigInteger) big).doubleValue().

As in the first example, this modification affects the database layout file: At the position where the Salary column of the Taeacher table is described, the value of the entry typeDesc is changed from BigIntegerni20v to d!i8 and the no longer needed entry

   typeFactoryClassName = example.db.BigIntegerType

is removed.

Opening the modified database as a weakly typed database is no problem. However, opening the database as a strongly typed database throws an exception. The reason is that the TeacherTable class defines the Salary column still to be of type BigInteger:

   public static final Column<BigInteger> SALARY =
                CL.create(new BigIntegerType(20, NULLABLE));

In order for the modified database to match the table logic again, we have to replace this declaration with

   public static final Column<Double> SALARY =
                                       CL.ofDouble(NO_NULL);

But what about existing client code that still assumes the Salary column to be of type BigInteger? Well, due to the strong typing of the column, this mismatch is conveniently reported to the programmer at compile time. (See the discussion in Section Strongly and Weakly Typed of Chapter Opening.)

Besides the methods addTable and modifyColumn presented here, the Refactor class includes the methods dropTable, dropColumn, and insertColumn. For further information consult the javadoc.

We also refer to the javadoc regarding the methods nobsRowRef, nobsOutrowPtr, and nobsRefCount, which make it possible to fundamentally re-dimension a table. The KeyFigures class prints for each table the values of these parameters and by how much they can be reduced given the amount of data currently stored in the database.

Checking Integrity

ACDP relies on the computer's file system. If ACDP behaves strangely, a reason may be that changes were made to database files from outside of ACDP, harming the integrity of the database. (The individual integrity constraints are highly technical in nature. Their explanation is beyond the scope of this tutorial.)

The method boolean ACDP.verify(Path layoutFile, boolean fix, boolean report) checks the integrity of the database specified by the layoutFile argument and, if fix is set to true and there is an integrity violation, ACDP tries to repair the database. For example, to check the integrity of the Edu database invoke

   ACDP.verify(Paths.get("dbDir/layout"), false, true);

If no integrity violations are detected the verify method returns true and prints the following report:

   Checking database "Edu" ...
      Checking table "Course" (1 of 4) ...
      Completed - No violations detected.
      Checking table "Student" (2 of 4) ...
      Completed - No violations detected.
      Checking table "Teacher" (3 of 4) ...
      Completed - No violations detected.
      Checking table "Contact" (4 of 4) ...
      Completed - No violations detected.
   Completed - No violations detected.