Package acdp

Interface Database

All Superinterfaces:
AutoCloseable
public interface Database extends AutoCloseable
Defines the database and provides the open(java.nio.file.Path, int, boolean, acdp.design.ICipherFactory) method.

A database comprises a collection of tables. The configuration of a database is contained in the database layout.

An instance of this class can be obtained by opening the database as a weakly typed database (see below) and applying the following usage pattern: 

 try (Database db = Database.open(...) {
    Table myTable = db.getTable("MyTable");
    do something with myTable
 }

Weakly and Strongly Typed

A database can be opened as a weakly typed or strongly typed database. An instance of this class is called a weakly typed database because the first access to a specific table of the database can only be done by the name of the table or by its position within the list of table layouts contained in the database layout, see the getTable(String) and getTables() methods. Changing the name of the table or changing the position of the table in the list of table layouts, without adapting the code on the client side, breaks the client code. Even worse, the compiler has no chance to detect such inconsistencies. Another drawback is that the tables of a weakly typed database are themselves weakly typed. (See section "Weakly and Strongly Typed" in the Table interface description to learn about weakly and strongly typed tables.)

In contrast to this, a strongly typed database is an instance of a class extending the CustomDatabase class. An elaborated custom database class can provide tailor-made methods for dealing with all kinds of aspects of cross-entity-specific persistence management. Opening a database as a strongly typed database creates its tables as strongly typed tables. Doing something with a particular table of a strongly typed database is simply doing it with the appropriate table instance which is an instance of a class extending the CustomTable class.

Opening a database as a weakly typed database can be done from the information saved in the database layout only. Therefore, ACDP does not need to know of a custom database class or of any custom table classes in order to open a database as a weakly typed database. Typically, a database is opened as a weakly typed database if there is no need for making any reference to the concrete entities the tables represent, for example, consider a tool which displays the persisted data of an arbitrary database without making any reference to a concrete entity.

WR and RO Database

A WR database (RO database) is a database where all of its tables are associated with a WR store (RO store), respectively. See section "Store" in the description of the Table interface to learn more about these two types of store. In a WR database that is not write protected, the database's tables support insert, delete and update operations. In an RO database or a write protected WR database such operations are not supported. Furthermore, a WR database consists of several files, including a separate file containing the layout of the database, whereas an RO database is backed by just one database file. Invoking the createRO(java.nio.file.Path, acdp.design.ICipherFactory) method converts a WR database to an RO database.

Units

An ACDP unit can be compared to the well-known database transaction. The main differences are:
  • Transactions running in different threads may interleave. Units, on the other hand, act as a monitor: Units in different threads are executed serially.
  • Nested units work as expected: If a sequence of write operations needs to be rolled back and this sequence of write operations contains other sequences of write operations in nested units then these sequences of committed write operations are rolled back as well. On the other hand, no write operation in any outer unit is affected if a sequence of write operations of a nested unit needs to be rolled back.
  • Reasonably, a unit contains at least one write operation. Units generally don't make sense if the database is read-only. To cope with view inconsistencies ACDP provides a different concept, the so called read zones (see below) which have an effect even across common transaction boundaries. This is important because isolation ("ACID - Atomicity, Consistency, Isolation, Durability") can be an issue beyond the scope of a transaction.
  • In a multi-threaded environment, transactions may be automatically rolled back at any time "by the system" due to deadlock resolution, leaving the client in the delicate situation to restart the transaction (tricky in praxis!) or abort the process. In contrast, a client of an ACDP database never has to worry about restarting a unit due to an intervention of the system.

A write operation, that is, an insert, delete, update, updateAll, updateAllSupplyValues or updateAllChangeValues operation, executed outside a unit is called a Kamikaze write. Kamikaze writes outperform write operations executed within a unit but their effects on the database can not be rolled back. Nevertheless, ACDP takes care that Kamikaze writes are safe for use in a multi-threaded environment. A typical use case of Kamikaze writes is the initial filling of an empty table with a large amount of data.

Read the description of the Unit interface to learn more about units and the common usage pattern.

Read Zones

A read-only operation, that is, a get, iterate, numberOfRows, refCount or a terminal operation of a stream pipeline can be invoked within a unit or a read zone to ensure that no concurrent writes are taken place in the database while the operation is being executed. Several read-only operations can be invoked within a read zone to cope with all kinds of view inconsistencies. Read the description of the ReadZone interface to learn more about read zones and the common usage pattern.

Service Operations

Besides those regular write and read database operations mentioned in the sections "Units" and "Read Zones" above, ACDP provides several service operations, most of them available only for a WR database.
Level 0
These are read-only service operations providing meta information about a particular table or the database. They can be invoked within a read zone or a unit in order to prevent any writes while the service operation is running.

Complete listing: all methods declared in the inner interfaces of the Information interface. Instances of classes implementing the DatabaseInfo and TableInfo interfaces can be obtained by invoking the info() and Table.info() methods, respectively.

Level 1
These are service operations that, as those on level 0, do not change the database but generate and save persistent data outside the database. Since they run within a read zone by default, invoking them inside a read zone or a unit has no effect.

Complete listing: createRO(java.nio.file.Path, acdp.design.ICipherFactory), zip(int, boolean, java.io.OutputStream).

Level 2
These are service operations that remove data from the backing files of a particular table.

They cannot be invoked inside a read zone or a unit and their effects onto the data persisted in the target table and in any other involved tables cannot be reverted. The integrity of the target table and of any involved tables may be harmed if a system crash occurs while a service operation on this level is running.

Complete listing: Table.truncate(), Table.compactVL(), Table.compactFL().

Level 3
Operations changing the structure of the database or which involve the modification of the internal format of some tables' backing files are level 3 service operations.

Unlike the other service operations, service operations on this level cannot be invoked within a session: The database must be closed at the time a level 3 service operation is invoked.

As with a service operation on level 2, the effects onto the data persisted in the database when running a service operation on level 3 cannot be reverted. The integrity of the database may be harmed if a running service operation on this level throws an exception due to any reason or if a system crash occurs. It may therefore be a good idea to backup the database before executing a service operation on this level.

Complete listing: All methods declared in the Refactor class.

All but the level 0 service operations are synchronized. This means that service operations beyond level 0 cannot interfere with any other database operation running in parallel.

Closing

Never forget to close the database when you are done with it. Since this interface extends the AutoCloseable interface you can apply the try-with-resources statement, see the code snippet above.

Zipping

The database can be packed into a single zip-archive-file by invoking the zip(int, boolean, java.io.OutputStream) method. Conversly, unzipping the generated zip-archive-file restores the database.

Durability

Mass storage devices, especially non-removal ones, usually apply caching strategies that defer writing data to the storage medium to an unknown later time. In case of a system crash this may lead to a loss of data. On the other hand, committed changes are expected to be durable even in the case of a system crash. A common way to cope with this problem is to keep a transaction log: After a system crash has occurred all committed transactions relative to a checkpoint can be redone with the help of the transaction log. To work properly, the transaction log must be saved on stable storage. However, most computer disk drives are not considered stable storage. Although ACDP logs any before data for rolling back an uncommitted unit, it does not log any after data. Instead, ACDP provides a switch in the database layout labeled forceWriteCommit. If set equal to "on" ACDP forces any data changes to be materialized at the time a sequence of write operations is committed. While this method guarantees durability in the case of a system crash even in an environment that lacks stable storage, it deteriorates performance because it interferes with disk controlling policies. This is why it can be turned off. If this feature is turned off then changes are forced being materialized not until the database is closed. However, you can always force changes being materialized by invoking the forceWrite() method. Note that Kamikaze writes (see section "Units" above) behave insensitive to the position of this switch or to an invocation of the forceWrite() method: Changes made with Kamikaze writes are never forced being materialized, not even when the database is closed. (At the time of writing this description, non-volatile RAM-based mass storage devices are being discussed, which presumably let the user find out if data sent to such a device has been successfully materialized.)

There should be no need for clients to implement this interface.

Author:
Beat Hörmann

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Closes this database and releases any system resources associated with it.
    void
    createRO(Path roDbFilePath, ICipherFactory cipherFactory)
    Creates an RO database from the data stored in this database, which must be a WR database.
    void
    forceWrite()
    Forces any changes to this database being materialized.
    Table
    getTable(String tableName)
    Returns the table with the specified name.
    Table[]
    getTables()
    Returns all tables of this database.
    boolean
    hasTable(String name)
    Tests if this database has a table with the specified name.
    Information.DatabaseInfo
    info()
    Returns the information object of this database.
    String
    name()
    Returns the name of this database.
    static Database
    open(Path mainFile, int opMode, boolean writeProtect, ICipherFactory cipherFactory)
    Opens the database as a weakly typed database.
    ReadZone
    openReadZone()
    Opens a read zone.
    Unit
    openUnit()
    Opens a unit.
    String
    toString()
    As the name() method, returns the name of this database.
    void
    zip(int level, boolean home, OutputStream out)
    Creates a zip archive containing the files related to this database and writes it to the specified output stream, leaving the output stream open.

  • Method Details

    • open

      static Database open(Path mainFile, int opMode, boolean writeProtect, ICipherFactory cipherFactory) throws NullPointerException, IllegalArgumentException, InvalidPathException, ImplementationRestrictionException, OverlappingFileLockException, CreationException
      Opens the database as a weakly typed database.

      As long as the database is empty, or as soon as it is empty again, the type of encryption can be defined (or redefined) with the cipherFactory argument.

      It is a good idea to treat the returned database instance as a shared singleton.

      Opening the same database (having the same main database file) more than once is possible if the database is an RO database. (Read section "WR and RO Database" of this interface description to learn more about the two kinds of databases.) However, there are limitations if the database is a WR database: Opening a WR database more than once within the same process is not possible whereas opening a WR database more than once each time in a different process is possible if the WR database is opened in all processes with write protection turned on and provided that the operating system supports shared locks. (If it were allowed to open the same WR database in one process with write protection turned off and in another process with write protection turned on, this would compromise the concept of the read zone.)

      Parameters:
      mainFile - The main database file, not allowed to be null. If the database is a WR database then the main file is identical to the layout file. Otherwise, the database is an RO database and the main file is identical to the one and only one RO database file.
      opMode - The operating mode of the database. If the value is equal to zero then the backing table files are immediately closed as soon as they become idle. If the value is positive and equal to n then the backing table files are closed max(10, n) milliseconds after they become idle or when the database is closed. If the value is equal to -1 then the backing table files once opened remain open until the database is closed. If the value is equal to -2 then the compressed content of an RO database is completely mapped into memory. If the value is equal to -3 then the uncompressed content of an RO database is completely mapped into memory. A value equal to zero is recommended only if the database is a WR database and the operating system can't sustain a bunch of files all being opened at the same time. In a server environment, a positive value is recommended or, provided that the database is an RO database and there is enough memory available, a value equal to -2 (less memory required, less fast) or -3 (more memory required, faster). Note that a value equal to -2 or equal to -3 raises an IllegalArgumentException if the database is a WR database.
      writeProtect - Protects a WR database from being modified. If set to true then no data of the database can be inserted, deleted or updated. Note that this parameter has no effect if the database is an RO database. A write protected database needs less system resources than a writable database.
      cipherFactory - The cipher factory or null if no encryption is required.
      Returns:
      The database, never null.
      Throws:
      NullPointerException - If mainFile is null.
      IllegalArgumentException - If opMode is less than -3 or if opMode is equal to -2 or -3 and the database is a WR database.
      InvalidPathException - If the database layout contains an invalid file path. This exception never happens if the database is an RO database.
      ImplementationRestrictionException - If a table has too many columns needing a separate null information. This exception never happens if the database is an RO database.
      OverlappingFileLockException - If the database was already opened before in this process (Java virtual machine) or in another process that holds a lock which cannot co-exist with the lock to be acquired. This exception never happens if the database is an RO database.
      CreationException - If the database can't be created due to any other reason including problems with the database layout, problems with any of the tables' sublayouts, problems regarding encryption and problems with the backing files of the database.

    • name

      String name()
      Returns the name of this database.
      Returns:
      The name of this database, never null and never an empty string.

    • info

      Information.DatabaseInfo info()
      Returns the information object of this database.
      Returns:
      The information object of this database, never null.

    • hasTable

      boolean hasTable(String name)
      Tests if this database has a table with the specified name.
      Parameters:
      name - The name.
      Returns:
      The boolean value true if and only if this database has a table with the specified name.

    • getTable

      Table getTable(String tableName) throws IllegalArgumentException
      Returns the table with the specified name.
      Parameters:
      tableName - The name of the table as written in the database layout.
      Returns:
      The table with the specified name, never null.
      Throws:
      IllegalArgumentException - If this database has no table with the specified name.

    • getTables

      Table[] getTables()
      Returns all tables of this database.

      The order in which the tables appear in the returned array is equal to the order in which the tables appear in the database layout.

      Returns:
      The tables of this database, never null and never empty.

    • openUnit

      Unit openUnit() throws UnitBrokenException, ShutdownException, ACDPException
      Opens a unit. Invoking this method inside a unit opens a nested unit.

      Use this method as follows: 

       // db instance of Database or CustomDatabase, not null.
 // Precondition: db.info().isWritable()
 try (Unit u = db.openUnit()) {
    ...
    u.commit();
 }
      Note that calling this method in a read-only database throws an exception.

      For more details consult section "Units" of this interface description.

      Returns:
      The unit, never null.
      Throws:
      UnitBrokenException - If the unit is broken.
      ShutdownException - If this database is closed.
      ACDPException - If this database is read-only or if this method is invoked within a read zone.

    • openReadZone

      ReadZone openReadZone() throws ShutdownException
      Opens a read zone. Invoking this method inside a read zone opens a nested read zone.

      Use this method as follows: 

       // db instance of Database or CustomDatabase, not null.
 try (ReadZone rz = db.openReadZone()) {
    ...
 }
      Apart from returning an instance of ReadZone, this method has no effect if this database is read-only.

      For more details consult section "Read Zones" of this interface description.

      Returns:
      The read zone, never null.
      Throws:
      ShutdownException - If this database is closed. This exception never happens if this database is read-only.

    • createRO

      void createRO(Path roDbFilePath, ICipherFactory cipherFactory) throws UnsupportedOperationException, NullPointerException, ImplementationRestrictionException, ShutdownException, CreationException, CryptoException, IOFailureException
      Creates an RO database from the data stored in this database, which must be a WR database.

      Note that existing references to rows of a particular table are valid only in the created RO database if the FL file space of the table in the original WR database has no gaps.

      If you open the WR database just for the purpose of invoking this method then it is a good idea to open the database as a weakly typed, write protected database and setting the operating mode to -1.

      The implementation of this method is such that concurrent writes can not take place while this method is running. It is therefore safe to invoke this method anytime during a session.

      Parameters:
      roDbFilePath - The path of the newly created RO database file. The value must be the path of a non-existing file.
      cipherFactory - The cipher factory or null if no encryption is required.
      Throws:
      UnsupportedOperationException - If this database is an RO database.
      NullPointerException - If roDbFilePath is null.
      ImplementationRestrictionException - If at least one of the tables of the WR database has more than Integer.MAX_VALUE rows or if the length of the byte representation of a stored column value of a table of the WR database exceeds Integer.MAX_VALUE or if at least one of the tables of the WR database has too many columns or if the number of row gaps in at least one of the tables of the WR database is greater than Integer.MAX_VALUE.
      ShutdownException - If this database is closed.
      CreationException - If creating the RO crypto object fails or if computing the cipher challenge fails. This exception never happens if cipherFactory is null.
      CryptoException - If decrypting the byte representation of a stored column value of a table of the WR database fails. This exception never happens if the WR database does not apply encryption.
      IOFailureException - If the specified file already exists or if another I/O error occurs.

    • zip

      void zip(int level, boolean home, OutputStream out) throws NullPointerException, IllegalArgumentException, ShutdownException, IOFailureException
      Creates a zip archive containing the files related to this database and writes it to the specified output stream, leaving the output stream open.

      With file being an instance of File and level as well as home set to some reasonable values, executing 

       try (FileOutputStream fos = new FileOutputStream(file);
      BufferedOutputStream bos = new BufferedOutputStream(fos)) {
    zip(level, home, bos);
 }
      creates a zip archive file.

      In case of an RO database the main file is the only file related to the database. Since the main file of an RO database is already quite compact, zipping an RO database generally does not make sense.

      If the home argument is set to false then the files appear in the resulting zip archive with the path information as contained in the database layout with the exception of the main file itself which appears with its file name only. (Recall that in a WR database the main file and the layout file, hence, the file containing the database layout, are identical.)

      If the home argument is set to true then the same file structure is created as if the home argument was set to false but with a single root directory having a name equal to the name of the main directory. (The main directory is the directory housing the main file.)

      For example, if the main file "v" has a path equal to "/a/b/v" (or "C:\a\b\v") and some other file "w" of the database has a path equal to "/a/b/e/w" (or "C:\a\b\e\w") with b denoting the main directory of the database then "v" and "w" will appear in the zip archive as "v" and "e/w", respectively, provided that the home argument is equal to false. If the home argument is equal to true then "v" and "w" will appear in the zip archive as "b/v" and "b/e/w", respectively.

      The database can easily be restored by unzipping the resulting zip archive to any directory. However, the database can be opened without any prior modifications only if the following two conditions are satisfied:

      • All file paths contained in the database layout are paths relative to the main directory.
      • No path points to a file residing outside the main directory, that is, no file is located outside the main directory. (Via the ".." path element a path relative to the main directory can point to a file outside of the main directory. Applying the ".." path element is perfectly okay as long as the path points to a file inside the main directory.)

      The implementation of this method is such that concurrent writes can not take place while this method is running. It is therefore safe to invoke this method anytime during a session.

      Parameters:
      level - the compression level or -1 for the default compression level. Valid compression levels are greater than or equal to zero and less than or equal to nine.
      home - The information whether all the files should be unzipped into a single directory with a name equal to the name of the main directory.
      out - The output stream, not allowed to be null.
      Throws:
      NullPointerException - If out is null.
      IllegalArgumentException - If the compression level is invalid.
      ShutdownException - If this database is closed.
      IOFailureException - If an I/O error occurs.

    • forceWrite

      void forceWrite() throws IOFailureException
      Forces any changes to this database being materialized. This method has no effect if this database is read-only or if the switch in the database layout labeled forceWriteCommit is set equal to "on".
      Throws:
      IOFailureException - If an I/O error occurs.

    • close

      void close() throws IOFailureException
      Closes this database and releases any system resources associated with it.

      If this database is already closed then invoking this method has no effect.

      Specified by:
      close in interface AutoCloseable
      Throws:
      IOFailureException - If an I/O error occurs.

    • toString

      String toString()
      As the name() method, returns the name of this database.
      Overrides:
      toString in class Object
      Returns:
      The name of this database, never null and never an empty string.