Merge: Merging QofBook structures
[Query Object Framework]


Data Structures

struct  QofBookMergeRule
 One rule per entity, built into a single GList for the entire merge. More...
struct  QofBookMergeData
 mergeData contains the essential context data for any merge. More...

Files

file  qofbookmerge.h
 API for merging two QofBook structures with collision handling.

Enumerations

enum  QofBookMergeResult {
  MERGE_UNDEF, MERGE_ABSOLUTE, MERGE_NEW, MERGE_REPORT,
  MERGE_DUPLICATE, MERGE_UPDATE, MERGE_INVALID
}
 Results of collisions and user resolution. More...

qof_book_merge API

typedef void(* QofBookMergeRuleForeachCB )(QofBookMergeData *, QofBookMergeRule *, guint)
 Definition of the dialogue control callback routine.
QofBookMergeDataqof_book_merge_init (QofBook *importBook, QofBook *targetBook)
 Initialise the QofBookMerge process.
void qof_book_merge_rule_foreach (QofBookMergeData *mergeData, QofBookMergeRuleForeachCB callback, QofBookMergeResult mergeResult)
 Dialogue Control Callback.
QofBookMergeDataqof_book_merge_update_result (QofBookMergeData *mergeData, QofBookMergeResult tag)
 called by dialogue callback to set the result of user intervention
gint qof_book_merge_commit (QofBookMergeData *mergeData)
 Commits the import data to the target book.
void qof_book_merge_abort (QofBookMergeData *mergeData)
 Abort the merge and free all memory allocated by the merge.

Detailed Description

Collision handling principles.

  1. Always check for a GUID first and compare. qofbookmerge only accepts valid QofBook data and therefore ALL objects in the import book will include valid GUID's.
  2. If the original import data did not contain a GUID (e.g. an external non-GnuCash source) the GUID values will have been created during the import and will not match any existing GUID's in the target book so objects that do not have a GUID match cannot be assumed to be MERGE_NEW - parameter values must be checked.
  3. If import contains data from closed books, store the data from the closed books in the current book as active. i.e. re-open the books.

More information is at http://code.neil.williamsleesmill.me.uk/

Each foreach function uses g_return_if_fail checks to protect the target book. If any essential data is missing, the loop returns without changing the target book. Note that this will not set or return an error value. However, g_return is only used for critical errors that arise from programming errors, not for invalid import data which should be cleaned up before creating the import QofBook.

Only qof_book_merge_update_result and qof_book_merge_commit return any error values to the calling process. qof_book_merge_init returns a pointer to the QofBookMergeData struct - the calling process needs to make sure this is non-NULL to know that the Init has been successful.

Todo:
Prior to libqof2 rationalise internal variable names.

Typedef Documentation

Definition of the dialogue control callback routine.

All MERGE_REPORT rules must be offered for user intervention using this template.
Commit will fail if any rules are still tagged as MERGE_REPORT.

Calling processes are free to also offer MERGE_NEW, MERGE_UPDATE, MERGE_DUPLICATE and MERGE_ABSOLUTE for user intervention. Attempting to query MERGE_INVALID rules will cause an error.

For an example, consider test_rule_loop, declared as:

void test_rule_loop(QofBookMergeData *mergeData, QofBookMergeRule *rule, guint remainder);
void test_rule_loop(QofBookMergeData *mergeData, QofBookMergeRule *rule, guint remainder)
{
g_return_if_fail(rule != NULL);
g_return_if_fail(mergeData != NULL); printf("Rule Result %s", rule->mergeType);
qof_book_merge_update_result(mergeData, rule, MERGE_UPDATE);
}

The dialogue is free to call qof_book_merge_update_result in the loop or at the end as long as the link between the rule and the result is maintained, e.g. by using a GHashTable.
The parameters are:

  • data : pointer to the QofBookMergeData metadata context returned by init.
  • rule : pointer to the QofBookMergeRule that generated the collision report
  • remainder : guint value returned from g_slist_length for the number of other rules remaining with the same result. This might be useful for a progress dialogue, it might not. When updating MERGE_REPORT, remainder must equal zero before calling qof_book_merge_commit or the import will abort.

If the dialogue sets any rule result to MERGE_INVALID, the import will abort when qof_book_merge_commit is called. It is the responsibility of the calling function to handle the error code from qof_book_merge_commit, close the dialogue and return. The merge routines in these files will already have halted the merge operation and freed any memory allocated to merge structures before returning the error code. There is no need for the dialogue process to report back to QofBookMerge in this situation.

Definition at line 321 of file qofbookmerge.h.


Enumeration Type Documentation

Results of collisions and user resolution.

All rules are initialised as MERGE_UNDEF. Once the comparison is complete, each object within the import will be updated.

MERGE_ABSOLUTE, MERGE_NEW, MERGE_DUPLICATE and MERGE_UPDATE can be reported to the user along with all MERGE_REPORT objects for confirmation. It may be useful later to allow MERGE_ABSOLUTE, MERGE_NEW, MERGE_DUPLICATE and MERGE_UPDATE to not be reported, if the user sets a preferences option for each result. (Always accept new items: Y/N default NO, ignores all MERGE_NEW if set to Y etc.) This option would not require any changes to qofbookmerge.

MERGE_NEW, MERGE_DUPLICATE and MERGE_UPDATE are only actioned after conflicts are resolved by the user using a dialog and all MERGE_REPORT objects are re-assigned to one of MERGE_NEW, MERGE_DUPLICATE or MERGE_UPDATE. There is no automatic merge, even if no entities are tagged as MERGE_REPORT, the calling process must still check for REPORT items using qof_book_merge_rule_foreach and call qof_book_merge_commit.

MERGE_INVALID data should be rare and allows for user-abort - the imported file/source may be corrupted and the prescence of invalid data should raise concerns that the rest of the data may be corrupted, damaged or otherwise altered. If any entity is tagged as MERGE_INVALID, the merge operation will abort and leave the target book completely unchanged.

MERGE_ABSOLUTE is only used for a complete match. The import object contains the same data in the same parameters with no omissions or amendments. If any data is missing, amended or added, the data is labelled MERGE_UPDATE.

Every piece of data has a corresponding result. Only when the count of items labelled MERGE_REPORT is equal to zero are MERGE_NEW and MERGE_UPDATE items added to the existing book.
MERGE_DUPLICATE items are silently ignored. Aborting the dialogue/process (by the user or in a program crash) at any point before the final commit leaves the existing book completely untouched.

Enumerator:
MERGE_UNDEF  default value before comparison is made.
MERGE_ABSOLUTE  GUID exact match, no new data - ignore
MERGE_NEW  import object does not exist in the target book - add
MERGE_REPORT  import object needs user intervention - report
MERGE_DUPLICATE  import object with different GUID exactly matches existing GUID - ignore
MERGE_UPDATE  import object matches an existing entity but includes new or modified parameter data - update
MERGE_INVALID  import object didn't match registered object or parameter types or user decided to abort - abort

Definition at line 125 of file qofbookmerge.h.

00126 {
00127     MERGE_UNDEF,     
00128     MERGE_ABSOLUTE,  
00129     MERGE_NEW,       
00131     MERGE_REPORT,    
00132     MERGE_DUPLICATE, 
00134     MERGE_UPDATE,    
00136     MERGE_INVALID    
00138 } QofBookMergeResult;


Function Documentation

void qof_book_merge_abort ( QofBookMergeData mergeData  ) 

Abort the merge and free all memory allocated by the merge.

Sometimes, setting MERGE_INVALID is insufficient: e.g. if the user aborts the merge from outside the functions dealing with the merge ruleset. This function causes an immediate abort - the calling process must start again at Init if a new merge is required.

Definition at line 937 of file qofbookmerge.c.

00938 {
00939     QofBookMergeRule *currentRule;
00940 
00941     g_return_if_fail (mergeData != NULL);
00942     while (mergeData->mergeList != NULL)
00943     {
00944         currentRule = mergeData->mergeList->data;
00945         g_slist_free (currentRule->linkedEntList);
00946         g_slist_free (currentRule->mergeParam);
00947         g_free (mergeData->mergeList->data);
00948         if (currentRule)
00949         {
00950             g_slist_free (currentRule->linkedEntList);
00951             g_slist_free (currentRule->mergeParam);
00952             g_free (currentRule);
00953         }
00954         mergeData->mergeList = g_list_next (mergeData->mergeList);
00955     }
00956     g_list_free (mergeData->mergeList);
00957     g_slist_free (mergeData->mergeObjectParams);
00958     g_slist_free (mergeData->targetList);
00959     if (mergeData->orphan_list != NULL)
00960         g_slist_free (mergeData->orphan_list);
00961     g_hash_table_destroy (mergeData->target_table);
00962     g_free (mergeData);
00963 }

gint qof_book_merge_commit ( QofBookMergeData mergeData  ) 

Commits the import data to the target book.

The last function in the API and the final part of any QofBookMerge operation.

qof_book_merge_commit will abort the entire merge operation if any rule is set to MERGE_INVALID. It is the responsibility of the calling function to handle the error code from qof_book_mergeCommit, close the dialogue and return. qof_book_merge_commit will already have halted the merge operation and freed any memory allocated to all merge structures before returning the error code. There is no way for the dialogue process to report back to qof_book_merge in this situation.

qof_book_merge_commit checks for any entities still tagged as MERGE_REPORT and then proceeds to import all entities tagged as MERGE_UPDATE or MERGE_NEW into the target book.
This final process cannot be UNDONE!

Parameters:
mergeData the merge context, QofBookMergeData*
Returns:
  • -2 if any rules are tagged as MERGE_INVALID
    • mergeData will have been g_free'd).
    • note that this will be before any operations are done on the target QofBook.
  • -1 if mergeData is invalid or no merge has been initialised with qof_book_merge_init - the calling process must check the value of mergeData
  • +1 if some entities are still tagged as MERGE_REPORT - use qof_book_merge_update_rule and try again (mergeData is retained).
  • 0 on success - mergeData will have been freed.

Definition at line 998 of file qofbookmerge.c.

00999 {
01000     QofBookMergeRule *currentRule;
01001     GList *check, *node;
01002 
01003     g_return_val_if_fail (mergeData != NULL, -1);
01004     g_return_val_if_fail (mergeData->mergeList != NULL, -1);
01005     g_return_val_if_fail (mergeData->targetBook != NULL, -1);
01006     if (mergeData->abort == TRUE)
01007         return -1;
01008     check = g_list_copy (mergeData->mergeList);
01009     g_return_val_if_fail (check != NULL, -1);
01010     for (node = check; node != NULL; node = node->next)
01011     {
01012         currentRule = node->data;
01013         if (currentRule->mergeResult == MERGE_INVALID)
01014         {
01015             qof_book_merge_abort (mergeData);
01016             g_list_free (check);
01017             return (-2);
01018         }
01019         if (currentRule->mergeResult == MERGE_REPORT)
01020         {
01021             g_list_free (check);
01022             return 1;
01023         }
01024     }
01025     g_list_free (check);
01026     qof_book_merge_commit_foreach (qof_book_merge_commit_rule_loop,
01027         MERGE_NEW, mergeData);
01028     qof_book_merge_commit_foreach (qof_book_merge_commit_rule_loop,
01029         MERGE_UPDATE, mergeData);
01030     /* Placeholder for QofObject merge_helper_cb - all objects
01031        and all parameters set */
01032     while (mergeData->mergeList != NULL)
01033     {
01034         currentRule = mergeData->mergeList->data;
01035         g_slist_free (currentRule->mergeParam);
01036         g_slist_free (currentRule->linkedEntList);
01037         mergeData->mergeList = g_list_next (mergeData->mergeList);
01038     }
01039     g_list_free (mergeData->mergeList);
01040     g_slist_free (mergeData->mergeObjectParams);
01041     g_slist_free (mergeData->targetList);
01042     if (mergeData->orphan_list != NULL)
01043         g_slist_free (mergeData->orphan_list);
01044     g_hash_table_destroy (mergeData->target_table);
01045     g_free (mergeData);
01046     return 0;
01047 }

QofBookMergeData* qof_book_merge_init ( QofBook importBook,
QofBook targetBook 
)

Initialise the QofBookMerge process.

First function of the QofBookMerge API. Every merge must begin with init.

Requires the book to import (QofBook *) and the book to receive the import, the target book (QofBook *). Returns a pointer to QofBookMergeData which must be checked for a NULL before continuing.
Process:

  1. Invoke the callback qof_book_merge_foreach_type on every registered object class definition.
  2. Callback obtains the registered parameter list for each object type. This provides run time access to all registered objects and all object parameters without any changes to QofBookMerge - no registered object or parameter is omitted from any merge operation.
  3. Use qof_object_foreach to invoke the callback qof_book_merge_foreach, one object at a time on every instance stored in mergeBook. This is the first point where real data from the import book is accessed.
  4. qof_book_merge_foreach obtains the GUID for the object from the import book and runs the first check on the original book, checking for any exact GUID match. With the full parameter list, the rules for this object can be created. If there is a GUID match, the data in each parameter of the import object is compared with the same semantic object in the original book. If there is no GUID in the import object or no GUID match with the original book, the original book is searched to find a parameter match - checking for a MERGE_DUPLICATE result.
  5. qof_book_merge_compare sets the QofBookMergeResult of the comparison.
  6. Inserts the completed rule into QofBookMergeData::mergeList GSList.

Returns:
NULL in case of error, otherwise a QofBookMergeData* metadata context.

Definition at line 900 of file qofbookmerge.c.

00901 {
00902     QofBookMergeData *mergeData;
00903     QofBookMergeRule *currentRule;
00904     GList *check;
00905 
00906     g_return_val_if_fail ((importBook != NULL)
00907         && (targetBook != NULL), NULL);
00908     mergeData = g_new0 (QofBookMergeData, 1);
00909     mergeData->abort = FALSE;
00910     mergeData->mergeList = NULL;
00911     mergeData->targetList = NULL;
00912     mergeData->mergeBook = importBook;
00913     mergeData->targetBook = targetBook;
00914     mergeData->mergeObjectParams = NULL;
00915     mergeData->orphan_list = NULL;
00916     mergeData->target_table =
00917         g_hash_table_new (g_direct_hash, qof_book_merge_rule_cmp);
00918     currentRule = g_new0 (QofBookMergeRule, 1);
00919     mergeData->currentRule = currentRule;
00920     qof_object_foreach_type (qof_book_merge_foreach_type, mergeData);
00921     g_return_val_if_fail (mergeData->mergeObjectParams, NULL);
00922     if (mergeData->orphan_list != NULL)
00923         qof_book_merge_match_orphans (mergeData);
00924     for (check = mergeData->mergeList; check != NULL; check = check->next)
00925     {
00926         currentRule = check->data;
00927         if (currentRule->mergeResult == MERGE_INVALID)
00928         {
00929             mergeData->abort = TRUE;
00930             return (NULL);
00931         }
00932     }
00933     return mergeData;
00934 }

void qof_book_merge_rule_foreach ( QofBookMergeData mergeData,
QofBookMergeRuleForeachCB  callback,
QofBookMergeResult  mergeResult 
)

Dialogue Control Callback.

This function is designed to be used to iterate over all rules tagged with a specific QofBookMergeResult value.

Parameters:
callback external loop of type QofBookMergeRuleForeachCB
mergeResult QofBookMergeResult value to look up.
mergeData QofBookMergeData merge context.
Note : MERGE_NEW causes a new entity to be created in the target book at commit which is then assigned as the targetEnt of that rule. If mergeResult == MERGE_NEW, the rules returned by qof_book_merge_rule_foreach will have a NULL set for the targetEnt. This is because commit has not yet been called and no changes can be made to the target book. The calling process must handle the NULL targetEnt and NOT call any param_getfcn routines for the target entity. The import entity is available for display.

Uses qof_book_get_collection with the QofBookMergeRule::mergeType object type to return a collection of QofEntity entities from either the QofBookMergeData::mergeBook or QofBookMergeData::targetBook. Then uses qof_collection_lookup_entity to lookup the QofBookMergeRule::importEnt and again the qof_book_mergeRule::targetEnt to return the two specific entities.

Definition at line 1050 of file qofbookmerge.c.

01052 {
01053     struct QofBookMergeRuleIterate qiter;
01054     QofBookMergeRule *currentRule;
01055     GList *matching_rules, *node;
01056 
01057     g_return_if_fail (cb != NULL);
01058     g_return_if_fail (mergeData != NULL);
01059     currentRule = mergeData->currentRule;
01060     g_return_if_fail (mergeResult > 0);
01061     g_return_if_fail (mergeResult != MERGE_INVALID);
01062     g_return_if_fail (mergeData->abort == FALSE);
01063     qiter.fcn = cb;
01064     qiter.data = mergeData;
01065     matching_rules = NULL;
01066     for (node = mergeData->mergeList; node != NULL; node = node->next)
01067     {
01068         currentRule = node->data;
01069         if (currentRule->mergeResult == mergeResult)
01070             matching_rules = g_list_prepend (matching_rules, 
01071                 currentRule);
01072     }
01073     qiter.remainder = g_list_length (matching_rules);
01074     g_list_foreach (matching_rules, qof_book_merge_rule_cb, &qiter);
01075     g_list_free (matching_rules);
01076 }

QofBookMergeData* qof_book_merge_update_result ( QofBookMergeData mergeData,
QofBookMergeResult  tag 
)

called by dialogue callback to set the result of user intervention

Set any rule result to MERGE_INVALID to abort the import when qof_book_merge_commit is called, without changing the target book.

The calling process should make it absolutely clear that a merge operation cannot be undone and that a backup copy should always be available before a merge is initialised.

Recommended method: Only offer three options to the user per rule:

  1. Allow import data to be merged into target data
    • change MERGE_REPORT to MERGE_UPDATE
  2. Allow import data without an exact match to be added as new
    • change MERGE_REPORT to MERGE_NEW IF mergeAbsolute = FALSE
  3. Ignore import data and leave target data unchanged
    • change MERGE_REPORT to MERGE_ABSOLUTE or MERGE_DUPLICATE

Handle the required result changes in code: Check the value of qof_book_mergeRule::mergeAbsolute and use these principles:

To ignore entities tagged as:

  • MERGE_REPORT, you must check the value of mergeAbsolute.
    • if mergeAbsolute is TRUE, change MERGE_REPORT to MERGE_ABSOLUTE
    • if mergeAbsolute is FALSE, change MERGE_REPORT to MERGE_DUPLICATE
  • MERGE_NEW, set MERGE_DUPLICATE.
  • MERGE_UPDATE, you must check the value of mergeAbsolute.
    • if mergeAbsolute is TRUE, change MERGE_UPDATE to MERGE_ABSOLUTE
    • if mergeAbsolute is FALSE, change MERGE_UPDATE to MERGE_DUPLICATE

To merge entities that are not pre-set to MERGE_NEW, set MERGE_UPDATE.
Attempting to merge an entity when the pre-set value was MERGE_NEW will force a change back to MERGE_NEW because no suitable target exists for the merge.

To add entities, check mergeAbsolute is FALSE and set MERGE_NEW.
An entity only be added if mergeAbsolute is FALSE. Attempting to add an entity when mergeAbsolute is TRUE will always force a MERGE_UPDATE.

It is not possible to update the same rule more than once.

  1. MERGE_NEW is reserved for new objects and is only pre-set if all parameters, including GUID, have already failed to match any relevant object. qof_book_mergeCommit will create new entities for all rules tagged as MERGE_NEW.
    • if mergeAbsolute is TRUE and the user wants to import the data, requests to set MERGE_NEW will be forced to MERGE_UPDATE because an entity with that GUID already exists in the target book.
    • if MERGE_NEW is pre-set, requests to change to MERGE_UPDATE will be ignored because a new entity is needed.
  2. MERGE_UPDATE is reserved for existing objects - qof_book_mergeCommit will require a matching entity to update and will force a change to back to MERGE_NEW if none is known to exist, using the principle above.
  3. MERGE_INVALID will cause an abort of the merge process.
  4. MERGE_UNDEF and MERGE_REPORT cannot be set - the entity result will be unchanged.
  5. MERGE_DUPLICATE and MERGE_ABSOLUTE are handled identically but are semantically different - QofBookMergeRule::mergeAbsolute is used to dictate which to set:
    • if mergeAbsolute is TRUE but MERGE_DUPLICATE is requested, force a change to MERGE_ABSOLUTE.
    • if mergeAbsolute is FALSE but MERGE_ABSOLUTE is requested, force a change to MERGE_DUPLICATE.

qof_book_merge_commit only commits entities tagged with MERGE_NEW and MERGE_UPDATE results.
Entities tagged with MERGE_ABSOLUTE and MERGE_DUPLICATE results are ignored.

The calling process must check the return value and call qof_book_merge_abort(mergeData) if non-zero.

Parameters:
mergeData the merge context, QofBookMergeData*
tag the result to attempt to set, QofBookMergeResult
Returns:
-1 if supplied parameters are invalid or NULL, 0 on success.

Definition at line 966 of file qofbookmerge.c.

00968 {
00969     QofBookMergeRule *resolved;
00970 
00971     g_return_val_if_fail ((mergeData != NULL), NULL);
00972     g_return_val_if_fail ((tag > 0), NULL);
00973     g_return_val_if_fail ((tag != MERGE_REPORT), NULL);
00974     resolved = mergeData->currentRule;
00975     g_return_val_if_fail ((resolved != NULL), NULL);
00976     if ((resolved->mergeAbsolute == TRUE) && (tag == MERGE_DUPLICATE))
00977         tag = MERGE_ABSOLUTE;
00978     if ((resolved->mergeAbsolute == TRUE) && (tag == MERGE_NEW))
00979         tag = MERGE_UPDATE;
00980     if ((resolved->mergeAbsolute == FALSE) && (tag == MERGE_ABSOLUTE))
00981         tag = MERGE_DUPLICATE;
00982     if ((resolved->mergeResult == MERGE_NEW) && (tag == MERGE_UPDATE))
00983         tag = MERGE_NEW;
00984     if (resolved->updated == FALSE)
00985         resolved->mergeResult = tag;
00986     resolved->updated = TRUE;
00987     if (tag >= MERGE_INVALID)
00988     {
00989         mergeData->abort = TRUE;
00990         mergeData->currentRule = resolved;
00991         return NULL;
00992     }
00993     mergeData->currentRule = resolved;
00994     return mergeData;
00995 }


Generated on Mon Jul 13 05:15:16 2009 for QOF by  doxygen 1.5.9