Object Management

In this tutorial you will learn about how you can organize your TileDB arrays and key-value stores hierarchically into TileDB groups. We call a TileDB array, key-value store or group as TileDB object. We also discuss some auxiliary TileDB object management functions.

Warning

This TileDB feature is experimental. Everything covered here works great, but the APIs might undergo changes in future versions.

Full programs

Program

Links

object

objectcpp objectpy

TileDB groups

TileDB allows you to hierarchically organize your arrays and key-value stores in groups. A group is practically a directory with a special (empty) TileDB file __tiledb_group.tdb. This offers an intuitive and familiar way to store your various TileDB objects in persistent storage. You can create a group simply as follows:

Listing the my_group directory, you get the following:

$ ls -l my_group
total 0
-rwx------  1 stavros  staff    0 Jul  3 10:08 __tiledb_group.tdb

Note that you can hierarchically organize TileDB groups similar to your filesystem directories, i.e., groups can be arbitrarily nested in other groups.

Getting the object type

TileDB also allows you to check the object type as follows. If path does not exist or is not a TileDB array, key-value store or group, it is marked as “invalid”.

Listing the object hierarchy

TileDB offers various ways to list the contents of a group, even recursively in pre-order or post-order traversal, optionally passing a special callback function that will be invoked for every visited object. This is demonstrated in the code snippet below:

In the object code example, we initially create the following hierarchy:

my_group/
├── dense_arrays
│   ├── array_A
│   ├── array_B
│   └── kv
└── sparse_arrays
    ├── array_C
    └── array_D

The code snippet we provided above would print out the following for this hierarchy (where <cwd> is the full path of your current working directory):

Move/Remove objects

TileDB offers functions for renaming and removing TileDB objects. Note that these functions are “safe”, in the sense that they will not have any effect on “invalid” (i.e., non-TileDB) objects.

You can rename TileDB objects as follows:

Note

Moving TileDB objects across different storage backends (e.g., from S3 to local storage, or vice-versa) is currently not supported. However, it will be added in a future version.

You can remove TileDB objects as follows:

Running the object code example, we get the output shown below. Observe the listing after my_group got renamed to my_group_2 and my_group_2/dense_arrays, my_group_2/sparse_arrays/array_C got removed.

$ ls -l my_group_2/
total 0
-rwx------  1 stavros  staff    0 Jul  3 11:18 __tiledb_group.tdb
drwx------  4 stavros  staff  136 Jul  3 11:18 sparse_arrays
$ ls -l my_group_2/sparse_arrays/
total 0
-rwx------  1 stavros  staff    0 Jul  3 11:18 __tiledb_group.tdb
drwx------  4 stavros  staff  136 Jul  3 11:18 array_D
$ ls -l my_group_2/sparse_arrays/array_D/
total 8
-rwx------  1 stavros  staff  115 Jul  3 11:18 __array_schema.tdb
-rwx------  1 stavros  staff    0 Jul  3 11:18 __lock.tdb