• Implement server actions
  • Implement automated processes
  • Implement import/export/synchronization interfaces

Frequently XSLT based solutions are appropriate or even more appropriate than Java development (not covered here).

Development setup

  • Use release distribution or git (export) distribution
  • When using git distribution, perform bin/build.sh init jar to generate classes and jars
  • Open project in Eclipse or setup IntelliJ (or use vi…)
  • Possible to start server within Eclipse but not recommended
  • Start server locally (does run on macOS and Windows), needs Oracle
  • Use Remote Java Application Debug setup in Eclipse (or IntelliJ)

In launcher.xml enable remote debugging:

< jvmarg value = "-Xrunjdwp:transport=dt_socket,server=y,address=8003,suspend=n" enabled = "true" />

Note: Performance/Memory profiler like YourKit highly recommended!

< jvmarg value = "-agentpath:/Applications/YourKit_Java_Profiler_2015_build_15074.app/Contents/Resources/bin/mac/libyjpagent.jnilib=listen=all" enabled = "true" />

Modules

  • Are in the following directories:
    • censhare-product/censhare-Server/app/modules directory (default)
    • censhare-product/censhare-Custom/censhare-Server/app/modules (custom)
  • Consist of:
    • XML description file (“command-xml”)
    • Java (source) code
    • Java properties files for localization
  • Hot deployable, Java source code is compiled and loaded:
  • Use UI admin client to “refresh” or
  • Use shell admin client bin/AdminClient.sh -b refresh

Persistence Architecture

  • Relational database has tables
  • In Java need „objects“ and/or XML
    • Like to have Plain Old Java Object (POJO)
    • Use as data transfer object (DTO)
    • Serializable
    • Should support changes / transactions
    • Like to have XML

Object Relational Mapping (ORM)

Class stack:

  • AXml (abstract base class)
    • XML node (we are not using org.w3c.dom)
  • CXml / BXml implementation
    • CXml is object instance based
    • BXml is byte[] based (CDB, online channel)
  • DXml adds change tracking / transactions
  • DataObject adds (database) persistence
  • AssetBase (automatically generated from db-schema)
  • Asset (additional methods)

Memory demand for a 1,9 MB XML file

Object Relational Mapping (ORM)

  • When setting the asset name using the setName method, the corresponding XML node attribute gets updated:

// Asset asset = ...
asset.setName( "Asset name" ).setAnnotation( "comment" );
 
// In generated AssetBase class contains:
public static final String ATTR_NAME = "name" ;
 
public T setName(String value) {
     xml.setAttr(ATTR_NAME, value);
     return (T) this ;
}

< asset name = "Asset name" annotation = "comment" .... />

  • How to know what to tell the database?
  • Database wants insert / update / delete
  • In transactional „mode“ DXml intercepts and notes the changes.
  • corpus:dto_flags
    • new - must insert
    • delete - should delete
    • modified - must update
    • transactional - is in transactional mode
    • persistent - is in database already

asset.setName("Asset name");
< asset name = "Old name" new-val:name = "Asset name" .... corpus:dto_flags = "tm" />

  • How to set the transactional mode?
  • You can do this manually (see below)
  • But typically the persistence manager does that for you

Asset asset = new Asset();
asset.setName( "old name" );
asset.setTransactionalRecursive( true );
asset.setName( "new name" );
System.out.println(asset.xml());

< asset corpus:dto_flags = "tm" name = "old name" new-val:name = "new name" />

Typical pattern:

DBTransactionManager tm;
PersistenceManager pm;
AssetManagementService am;
 
tm = command.getTransactionManager();
pm = tm.getDataObjectTransaction().getPersistenceManager();
am = ServiceLocator.getStaticService(AssetManagementService. class );
 
tm.begin();
 
Asset asset = Asset.newInstance();
asset.setName( "new name" );
asset.setType(AssetType.PICTURE.getDBValue());
asset.setApplication( "default" );
 
pm.makePersistent(asset);
am.checkInNew(tm, asset);
 
tm.end();

tm = command.getTransactionManager();
pm = tm.getDataObjectTransaction().getPersistenceManager();
am = ServiceLocator.getStaticService(AssetManagementService. class );

Note: tm and pm have a one-to-one relationship („separation of concerns“…)

tm.begin();
 
Asset asset = Asset.newInstance();
asset.setName( "new name" );
asset.setType(AssetType.PICTURE.getDBValue());
asset.setApplication( "default" );
 
pm.makePersistent(asset);
am.checkInNew(tm, asset);
 
tm.end();

tm.begin

  • Starts a new transaction
  • Marks objects in pm as transactional

tm.begin();
 
Asset asset = Asset.newInstance();
asset.setName( "new name" );
asset.setType(AssetType.PICTURE.getDBValue());
asset.setApplication( "default" );
 
pm.makePersistent(asset);
am.checkInNew(tm, asset);
 
tm.end();

pm.makePersistent

  • Creates primary keys
  • Attaches object to pm (put’s it into pm)

tm.begin();
 
Asset asset = Asset.newInstance();
asset.setName( "new name" );
asset.setType(AssetType.PICTURE.getDBValue());
asset.setApplication( "default" );
 
pm.makePersistent(asset);
am.checkInNew(tm, asset);
 
tm.end();

tm.end

  • Performs SQL statements but no commit
  • Marks objects in pm as non-transactional
  • tm.commit additionally does SQL commit and issues events
  • tm.commit is done automatically on command completion
  • Objects stay in pm!

Common mistakes

  • pm is like a bag that keeps all objects
  • Some APIs attach (insert) assets to the pm, others don‘t. (E.g.: AssetCacheService cacheGetAsset vs. cacheGetAssetNoAttach)
  • For bulk changes you need to manage that manually to avoid an OutOfMemoryException (E.g. call tm.end and pm.reset)
  • New assets don‘t have new-val

Missing or too frequent am.update.

tm.begin();
 
asset.setName(newName);
if (asset.isNewDeletedOrModifiedRecursive())
     am.update(tm, asset);
 
tm.end();

  • The persistence layer does write the change in the database. But:

    • doesn’t know how to handle files added to an asset
    • doesn’t create asset events that notify the clients
    • cannot perform checkout / checkin
    • doesn’t update versioned asset_relation / asset_element_relation table

  • Cannot perform checkout / checkin in one step, must use tm.begin / tm.end

  • Events are posted with tm.commit(), don’t commit every single change!
  • To avoid deadlocks if working with multiple assets in one transaction the assets must be processed in ascending asset id order !!! (E.g. use TreeSet / TreeMap to collect assets first)

tm.begin();
Asset asset = am.checkOut(tm, Asset.newUn(id, Asset.VERSION_CURRENT));
tm.end();
 
tm.begin();
asset.setName(newName);
am.checkIn(tm, asset);
tm.end();

  • Use bulk operations for performance:
    • to get 10 assets using cacheGetAsset(DBTransactionManager tm, AssetCacheKey key): costs 10 x about 12 sql statements - if not using CDB
    • to get 10 assets using cacheGetAsset(DBTransactionManager tm, AssetCacheKey[] keys): costs about 12 sql statements - if not using CDB
  • If transaction is “dirty” – first SQL insert/update statement executed – CDB no longer used to allow for read after write

Notes on CDB

  • Stores current asset versions
  • Almost all queries for current versions are handled through it (if transaction not dirty)
  • All returned assets are checked to be „up to date“ using the ccn with a fallback to Oracle (use queryAssetsLazy if not required)
  • asset.ccn is a change control number
    • tcn incremented with each update
    • to allow for change detection without locking assets and updating their tcn (e.g. new asset relation)

asset.ccn = tcn + (select count(*) from asset_ccn_counter where id = asset_ccn_counter.asset_id

Asset API

  • Hidden in asset.struct() to avoid messy code completion suggestions

StorageItem si = asset.struct().getStorageItem(StorageItemKey.MASTER.getDBValue());
 
for (ParentAssetRel rel : asset.struct().getParentAssetRelIter(AssetRelType.VARIANT)) {
     long parentId = rel.getParentAsset();
     ...
}

Query API

  • Common API for embedded DB and Oracle
  • Compatibility for legacy implementations
  • XPath queries

Query Builder

  • Fluent interface for creating queries - with types (refactoring!)
  • Works together with schema created classes
  • Works stand alone!

QBuilder qb = new QBuilder(QSchemaDef.SCHEMA_DEF);
QAsset a1 = qb.table(QAsset. class );
QChildAssetRel r1 = qb.table(QChildAssetRel. class );
QSelect stm = qb.select()
     .table(a1)
     .join(r1, r1.join(qb, a1))
     .columns(a1.ID, a1.VERSION, a1.CURRVERSION, r1.HAS_UPDATE_CHILD_GEOMETRY)
     .where(
         qb.expr(a1.NAME, "Hello" )
     );
System.out.println(SQLQueryBuilder.toString(stm));
 
QBuilder qb = new QBuilder();
QTable t1 = qb.tableAdHoc( "something" );
QSelect stm = qb.select().table(t1).columns(t1.defString( "name" ));
System.out.println(SQLQueryBuilder.toString(stm));

Asset Query Service

StringBuilder sb = new StringBuilder();
AssetQueryService qservice = ServiceLocator.getStaticService(AssetQueryService. class );
AssetQBuilder qb = qservice.prepareQBuilder();
QAsset tAsset = (QAsset) qb.getMainTable();
// qb.setSqlFallback(true); // force oracle
 
QSelect select = qb.select(tAsset).
         where(qb.and(
                 qb.expr(tAsset.CURRVERSION, 0 ),
                 qb.expr(tAsset.ID, ">=" , 1000 ),
                 qb.expr(a.TYPE, "picture" ),
                 qservice.preparePermissions(qb, tm.getTransactionContext().getUserId())
                 )).orderBy(tAsset.NAME);
 
int count = 0 ;
for (Asset asset: qservice.queryAssets(select, tm)) {
     count++;
     if (command.isCanceled())
         return Command.CMD_CANCELED; // or throw Exception
     sb.append(asset.getId()); sb.append( "\t" );
     sb.append(asset.getName()); sb.append( "\n" );
     if (count % 1000 == 0 )
         logger.info( "at: " + count + ", " + asset);
}
logger.info( "done at: " + count);
cmdXml.put( "report" , sb.toString());

AssetAutomation

  • AssetAutomation allows combined code for server actions and automated processes
  • See modules.lib.asset_automation.AssetAutomationSample
  • All “AAxxx” modules are implemented that way