- Fixes for CegoBTreeManager::deleteBTree ( changed int type to
PageIdType for page vars )
and CegoBTreeManager::insertBTree ( propValue was not setup correctly
for root node propagation in some cases, which might lead to corrupt
btree structure )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
lfcbase:
- Added range check to Chain::toInteger method to catch overflow exception
cego:
- This version brings a complete redesign of low level page handling.
Instead of page references identified by fileId and pageid, a
database unique pageid is used now
This results in a complete reimplemtation of several low level
classes like CegoFileHandler, CegoBufferPool, Blob handling, etc.
Since pages are references by a single ( 64 bit ) id now, I expect
an increased performance behaviour over all database operations.
Most code modifications are done, code complies and basic
functionally works ( create tableset, create table, insert table )
- First performance analysis indicates a speedup of about 10% for
btree creation, so significant speedup for full table scans.
- All base checks passed, but there is still a page allocation leak
for table drops
- Functional tests with SysMT successful completed
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoFieldValue for fixed value handling.
In case of fixed value aggregations, the specified fixed dimension
should be kept. This has been done be analysing the value dimension
in the CegoFieldValue:add/sub/mu/div methods.
In CegoQueryHelper::prepareFieldValue, fixed values are checked now
for right dimension
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
The only reason to use post-stage is because the port needs to do
"things" at a later time, like some plist manipulation.
While there, fold post-install in do-install targets when they are
defined.
PR: 214780
Submitted by: mat
Exp-run by: antoine
Sponsored by: Absolight
- Fix in CegoSelect::getQueyId added, for the ordering clause asc
and desc ordering attributes were missing. This might lead to wrong
ordering results if query caching is used
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Changed CegoQueryHelper methods to static
- Fix for CegoBTreeManager and friends regarding small string values
( string(1) )
- Please note, that this fix invalidates btree format.
Btrees have to be recreated after this fix, databases need to
be exported and imported.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed a file handle leak for datafiles and logfiles ( CegoFileHandler,
CegoTableManager and CegoLogManager ) After tableset creation (
or start and stop ), file handles have not cleaned up properly.
This lead to file remove problems in case of a tableset drop.
This effect was detected by building and testing the MinGW64 port
Submitted by: lemke@lemke-it.com
- Fix in CegoClient for escape character handling in batch mode
- Fix in CegoClient for dumpfile generation. Default values handling corrected
- Join optimizer fix in CegoSelect, CegoAttrCond and CegoBtreeCursor.
join attributes are checked now in a more sophisticed way, which
leads to improved execution performace for advanced joins
PR: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoFunction::evalFieldValue, for functions date2str,
date2int, int2date, type casting to appropriate type has been added.
Without the cast, the functions might return wrong results
- Adaptions for CegoBlowTab to use CegoNet API
- Completion for blob methods in CegoNet
- Improved flow control in CegoClient ( based on ncurses )
- Optimization in CegoClient, just high level CegoNet API methods
are used now for client implementation. Missing methods have been
added to CegoNet API
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoNet::connect, the global vars __dateTimeFormat,
__currencySymbol and __decimalPoint are set up now to appropriate
defaults
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.21 -> 2.30.23
lfcbase:
- Improvements for BigDecimal class
All values are normalized now and stored now with non trailing zeros
cego:
- Fixes for cast operations regarding values for type fixed
- Fix in CegoClient, global vars __currencySymbol and __decimalPoint
still have not been set up correctly, done in the same way as for
CegoMain on server side
- Added verify005.sql to dbcheck suite to verify cast operations
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoProcedure::execute and CegoProcedure::getRetVal In the
castTo method, the type dimension was still not treated. This might
lead to wrong cast results especially for fixed values
- Fixed cgadm online help for import command, the nologging option
was not documented
- In CegoRecoveryManager, added tableset argument line argument for
calling the external log manager.
This makes it more easy in the log manager code, to implement
tableset dependent restore strategies
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.17 -> 2.30.19
lfcbase:
- Added File::rename method to be able to move files via file class
- Patch file File class, operator= and operator+= did not close the
file descriptor after copying / appending. This may cause a problem
for subsequent method calls on the instance. ( e.g. renaming or
removing )
This effect actually has just been observed for MinGW64 compiles
cego:
- Catched server termination in CegoTableManager::writeCheckpoint.
If log files could not be archived, the method does not recognized
server termination ( runstate = OFFLINE )
- Improvement added for log file shipping in CegoLogThreadPool::copyLog.
The archive log file is first written to a temporary file and if
finished, is moved than to its original file name. For the tape
management, this is an indication, that the file is completely
written and is ready for further processing
- The previous patch in lfcbase required a patch for
CegoLogThreadPool::copyLog. Since the archfile file was not closed
after trunc, the rename operation failed now. close method call was
added
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed for point-in-time recovery, Date format handling was not
done correectly
- Elimination of BUFFERPOOLHASHKEY. The bufferpool position is now
calculated based on the page offset of the corresponding datafile
( this information is provided by CegoFileHandler::getRegPageOffset )
This leads to a much better distribution and usage of the available
buffer pool pages
- Fix in CegoTableManager::createBTree, the creation of the btree object
must be done in a dedicated exception block, otherwise the btree
object is dropped, if it already exists.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.14 -> 2.30.15
lfcbase:
- Added statlock mutex in ThreadLock class for protecting statistic
counters. It seems, increment and decrement operators on atomic
values are no more thread safe ( Once it was, but I guess this is
a compiler issue )
Since the lock statistics can be disabled for production mode,
there is not performance impact caused by the additional mutex
calls
cego:
- Made __dateFormatList in CegoMain threadsafe
For the point-in-time recovery timestamp format, now the server
defined format is used ( variable CGPITFORMAT is obsolete )
- Added update sync mode. With this mode, the user can decide, if
an update or delete should be blocked until another concurrent
transaction is finished or return immediatly without updating
concurrent affected tuples
- Reactivation of rollback segment record locks in CegoLockHandler class.
In CegoObjectManager::insertData, in case of rollback segements
insert operations, the RB record lock is used instead of the
system record lock. This might avoid some deadlock situations.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added alter table modify command just for default values.
alter table t1 modify column a default 42;
- Improvement for hash calculation in CegoBufferPool::calcSegement.
For a better page spreading, the segment calculation was changed
to a more linear approach
- Added more cast cases to CegoField::castTo, still not completed
- Cast handling in CegoFieldValue::castTo completed.
This allows an any to any value casting, except lob values and
some unsupported datetime casts ( e.g. bool to datetime really
makes no sense )
- Fix in CegoBufferPool::poolInfo, the spread rate calculation was
not done correctly
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added cast improvements to CegoFieldValue::castTo method, which are
also checked now with check050
- Evaluation of locale to find out decimal point character. E.g.
for english locales this is ., for german this is ,. The character
is used for float value handling in CegoFieldValue methods.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoRecoveryManager::recoverTableSet, the lsn was incremented
which lead to leaks in the transaction log. This could result in
an incomplete recovery procedure
- Improvements for tableset recovery, external log manage field is
checked in CegoAdminThread::medRecover before restore procedure is
started.
- Fix in CegoLogManager::logAction, if online redo log is full, the
current log entry has to be written anyway, otherwise a checkpoint
is written and subsequent logAction call in CegoTableManager::logIt
might lead to duplicate entries in case of a crash recovery
- Fixes for query and table cache added (addEntry method was still not clean)
- Added tableset runstate CHECKPOINT to be set during a tableset
checkpoint is performed.
Since this is a critical section in terms of data consistency,
no system crash should occur while writing a checkpoint.
If so, the tableset now is marked with status CHECKPOINT and cannot be
used anymore ( tableset recovery from backup required )
- Patch in CegoDistManager::startTableSet, before starting redo log
file recovery, the tableset state has to be set to RECOVERY
Otherwise, occupied redo logs are not archived by the log manager.
This problem might occur in case of a crash recovery with archive
mode set to ON
- Patch in CegoQueryCache and CegoTableCache ( for each method addEntry )
The usedSize calculation was not done correctly for replaces entries
- Added table and cache used size information to show tableset admin command
This indicates the amount of memory, which is already allocated
for cached data
- Added synrchonization for query and table cache handling.
The lack of synchronization might lead to core dumps, if a cache
entry is freed but also be used by a query
So before cache cleanup, it has to be checked,
if the corresponding cache entry is still in use
If so, it has to be wait, until the cache is clean. For both
caches ( query and table), the similar logic is used
- Added improvement for query cache handling
If a stored function, which are used in the cached query, was
modified, the cache entry will be invalidated
For this, the CegoSelect::getTableList method was changed to
CegoSelect::getObjectList and also the used functions are evaluated
and added to this list.
- Adapted sysmtexp and sysmtimp X-port utilities for current schema definition
- Cache cleanup in CegoQuery class is performed now AFTER the
modifying action.Otherwise, it may appear, that a cache entry is
made after cache cleanup but also before the modifying action.
This may lead to wrong results retrieved from obsolete query cache
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed memory leak in CegoSelect::nextTuple.
Since the querycache is allocated by the CegoQueryCache instance, the
pCacheList variable has to be freed after calling the addEntry method.
Actually, the logic is not very smart, since the cache element is build up
two times...
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
The following query can cause a core dump
create table t1(a string(100));
insert into t1 values ( 'Hugo Habicht');
insert into t1 values ( 'Kim Kaiser');
select substr(a, 1, getpos(a, ' ')) from t1;
For return value length evaluation, the method CegoFieldValue::asInteger
is called on a null value which causes the dump. A fix has been
added, so the method returns a value of 0 in case of null values.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.2.3 -> 1.2.4
databases/cego: 2.30.3 -> 2.30.5
lfcbase:
- code format cleanup
lfcxml:
- code format cleanup
cego:
- Code optimizations tested. The performance improvements on
subselects are achieved as expected.
- Code optimization for join buffer handling. Now a double pointer
list is used to just keep references on CegoField lists. This should
avoid expensive memory copies in case of creating parent join buffers
for sub selects.
- code format cleanup
- Fix in CegoAdmMon, CGNOMONITOR definition was fixed, now defined
via configure
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
Changes:
- Fix in CegoBufferPool::bufferFix. Small leak, since last page for each
pool segment never could be used ( hashId calculation )
- Cosmetic syntax change for set permission and remove permission admin
command
- Small information fix for query cache information in show tableset
admin command
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.29.3 -> 2.30.2
Changes:
- Fix in CegoAction and CegoDbHandler to handle query abort requests correctly
( initiated by cgadm via abort db thread command )
- Fix in CegoDistCursor for query plan calculation ( join objects )
- Fix in CegoAdminThreadPool and CegoLogThreadPool for thread load
calculation
- Major change for tablecache and querycache management,
for both dedicated tableset entries are managed. Cache enabling,
cache size and num entries can be controlled now on tableset level.
- Fixes for date format handling. The format string whihc is specified
for the server is also used now for client formatting. This results
in some network protocol fixes for serial but also xml network
protocol.
- Changed default protocol from xml to serial. Date format handling
completely implemented
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.29.1 -> 2.29.3
Changes to lfcbase:
- Fix in Chain for truncleft, truncRight and cutTrailing. In case
of single character strings, the methods might return wrong results.
- Started with implementation of TextIndex class ( utility class
for full text index creation )
Changes to cego:
- Fixes for table and query cache handling
- Fix in CegoDistManager::dropDistObject, table and query cache
have been checked for cleanup, if a table is dropped
- Fix in CegoQuery.cc, added table and query cache cleanup for alter
and rename queries
- Fix in CegoClient to parse multi line statements correctly. Further
multi line merge token ( \ ) has been added to merge long string or
clob values. Now the following statements are possible:
insert into t1 values ( 1, 'this is\
a multi line \
message \
with four lines');
- Added include of string.h in CegoAction.c ( compile error occured
on Suse Linux )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.28.3 -> 2.29.1
databases/cegobridge: 1.1.1 -> 1.1.2
Please note: Updating cego databases needs an export/import
cycle for 2.28 -> 2.29
lfcbase:
- Fix in Datetime.cc, added include config.h, otherwiese the local
implementation of strptime ist used. This should just be done, if
the system API does not provide this call ( e.g. MINGW ) .
- Memory leak fixed in File::readLine
cego:
- Started with clob datatype implementation
- Fix in CegoSelect constructor, _cacheEnabled var was not initialized
in the decoding constructor, which might lead to core dumps in case
of recovery of complex with enabled querycache Fix in CegoAdminThread
constructor, disableAuth method for TableManger called, otherwise
tableset recovery might fail with auth error
- Required network protocol expansions done for clob handling
(putclob, getclob, delclob)
- More work on clob handling. A fix was done for update operations
on clob values in CegoTableManager::updateTuple method. Before
updating the tuple, all clob references must be cloned, otherwise
in case of untouched clobs, references are invalid. This approach
is brute force, better would be, to stay on the untouched clob
references. But in terms of transaction and recovery handling,
this is not trivial
- Very long strings are handled now in CegoAction and CegoClient
to also parse large clob values
- Modification in cgclt for raw mode, Now just plain data is listed,
with the command line option --raw=X, a separator token can be setup
as a column separator
cegobridge:
- fix build problem
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.1.8 -> 1.2.2
databases/cego: 2.27.2 -> 2.28.3
databases/p5-DBD-cego: bump portrevision
databases/cegobridge: bump portrevision
IMPORTANT: Upgrades of cego 2.27.x to 2.28.x databases has to be done by
export and import.
lfcbase:
- Adaptions made for build with msys2 / mingw64. Many windows
specific stuff could be eliminated, since mingw64 provides better
POSIX support ( e.g. pthreads, timer functions .. )
lfcxml:
- Fix in configure.ac to find lfcbase and for FreeBSD
cego:
- Fix in CegoSelect::CegoSelect, the pCacheList variable was not
initialized to 0 for all constructors. In case of system table
selects, the variable might be undefined which might lead to
segmetation faults.
- Changed synchronization level for btree creation. So the base
table can also be accessed while btree is created ( During creation
btree status is invalid )
This also allows to perform on online btree reorganization, since
a new btree can be build up in parallel
- Fix in CegoSystemObject::createSystemSpace, id values for bustat
table must start with id 1 instead of id 0, otherwise, bustat
information retrievel fails or may result in a core dump
( select * from $bustat )
- Added performance fix for CegoQueryHelper::decodeFVL, since the
tuple context information is already from the method called, the
tid, tastep and ts parameters have been remove from this method.
This saves some decoding time
- Please note, that version 2.28.x makes datafiles from older versions
incompatible.
An upgrade requires a xml tableset export of existing data and a
restore after succesful upgrade
- 2.28.x also contains many code cleanup fixes and some fixes for
btree handling.
- Fix in CegoTableManager::createBTree, in case of an exception
during build up, the allocated have to be cleanup up. For this the
a rollback method was introduced in class CegoBTreeManager.
Fix in CegoTableCursor, invalid btree objects have not been ignored
- Many adapations to make cego full 64bit ready also in combination
with MSYS2 / MINGW64. This resulted in change of many variables
from type long to type unsigned long long (type long is just 32 bit
for Windows ) With an appropriate MSYS2 / MINGW64 installation,
cego compiles proper and runs stable now.
Many thanks to the MSYS/MINGW developers for their amazing work !
- Several changes from type int to type unsigned long long
- Fix in CegoTableManager::deleteDataTable ( renamed to
deleteDataTableEntry )
Since we allow now parallel updates, the method must check if the
target tuple has already been touched by another transaction.
Otherwise, invalid double entries could occur in case of concurrent
updates on the same tuple.
If a concurrent transaction is detected, the method returns false.
This return value is used by the updateTuple method to decide,
if the subsequent insert operation must be performed.
- Since it is not enough jut checking the tuple header info for
update deletes, a record locking has to be added to avoid duplicates.
In this sense, record locking for setTupleInfo method call ( used
in CegoTableManager and CegoTransactionManager ) was added. For
this, the lockRecord and unlockRecord method have been reactivated
in CegoLockManager class.
This lock avoids, that in case of concurrent tuple updates duplicate
tuples are produced by an update.
- Fix in configure.ac to find lfcbase and lfcxml and for FreeBSD
Submitted by: lemke@lemke-it.com
- Fix for system tables in CegoAttrDesc::evalReferences, for system
tables the evalTableReferences method has to be called, otherwise
attributes for system tables can not be accessed
- Add dbCheck for Fetch & Cache ( check047 ), all base checks are
run now with querycache ON and OFF
- Fix in CegoSelect::nextTuple, the cache schema was not set up
properly, if nextTuple was just called with empty jfl. This happens
in case of procedure cursors, ( CegoProcFetch ), where the field
value list is for all method calls empty
- Fix for CegoExpr::getAggregationList(), missing break operations
in switch block. This bug might lead to error messages regarding
aggregation queries like
CGCLT > select sum(a) + 0 from t1;
Error : Invalid expression sum(t1.a) + 0 for aggregation
- Modification for admin command set tsroot. Also the base path for
all defined datafiles is set now to new tsroot
- Fix in CegoBTreeCursor and CegoBTreeManager. Several conditons
have been changed from fileId && pageId to fileId || pageId
This bug may results in invalid btree objects in case of btree
spawn over several datafiles ( the pageId might be 0 in the next
file )
- Fix in CegoGroupSpace::insertTuple, the grouping tuple has to be
searched using getTableAlias instead of getTableName, otherwise the
following query returns wrong results
select count(*), a from t1 group by a; --works
select count(*), a from t1 tx group by a; -- broken
A check for this has been added in dbcheck suite
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Introduced BTree Cache to speed up the creating of large btree
indizes. A BTree Cache test done with 100 million tuples went
from 20h to 2.6h to create the index.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.26.14 -> 2.26.16
lfcbase:
- Improvement for ListT template class. The end of the list is now
stored int _listEnd variable. This improves Insert performance,
since the end of the list can immediately accesed instead of tracking
through the whole list until the end is reached.
- Patch in ThreadLock, the mutex for lock statistics has been
removed, since there is no need for synchronization. Also errors
have not been checked for the pthread_rwlock_wrlock, which might
cause some deadlocks.
cego:
- Ugly bug fixed in CegoObjectManager::releaseDataPtrUnlocked.
The logical operator && must be replaced by ||, otherwise fix leak arises
- Fix in CegoSelect::nextTuple for query cache to avoid, that queries
including system tables are cached.
- Error message fixes in CegoXPorter
- Introduced bufferpool distribution rate information in show pool output.
This indicates, if the buffer pool is used in a balanced way.
The behaviour can be changed with the BUFFERPOOLHASHKEY parameter.
The list pool command now receives the pool information in several
chunk messages. This allows to print also very large buffer pool
configurations
- Introduced BUFFERPOOLHASHKEY as a database tuning parameter. Based
on this parameter, the corresponding bufferpool slot for a fileId,
pageId tuple is calculated. Depending on the configured datafile
size, it might be useful to adjust this parameter
- Cego Tera Test started. Approach is to load about 1TB of random
data through the whole stack ( via cgblow ). Actually we are running
with 19 parallel processes loading concurrently 4 different tables.
Estimated date to achieve TB-Size is in about one month.
- Changed command line argument --nolockstat to --lockstat. So the
default value is lockstat disabled, since this has impact to the
database performance
- Modification of hash function in CegoBufferPool to achieve a
better distribution for large buffer pool configurations
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.1.7 -> 1.1.8
databases/cego: 2.26.13 -> 2.26.14
lfcbase:
- Added kill method to class Process, added SigTest check program to
basecheck suite
- Fix in BigDecimal::operator < and > .. comparison was not done correctly.
- Added automake check suite in baseecheck directory ( derived from samples )
lfcxml:
- Added automake check suite to package
- Small fix for XMLSuite constructor for pInStream initialization
cego:
- Added encoding implementation for CegoCaseCond to allow case
conditions also for update operations like
update erp_disk
set location = case when substr(wwn,20,1) = '1' then 'MD' else 'GV' end;
- Cosmetic changes in CegoDefs, increased FILEMNG_MAXDATAFILE to 5000
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoAttrDesc::evalReferences. In case of join objects, the
refCount was not calculated correcty, since just the join object
was analysed but not the sub content object. For this, a new method
was introduced ( evalTableReferences ) and in case of join object,
evalReferences is called recursive for all sub content objects
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- cgsimload with 3 mio ops and 2 processes passed
- Fix in CegoBTreeCursor::getFirst, if pAttrCond == 0 and btree is empty,
the method call tries to claim a zero datapointer
- Reorganization of CegoDistManager and CegoTabelManager, all
methods, which are calling useObject/unuseObject are located in
CegoTableManager class now.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added more sophisticated corruption check for btrees
( CegoBTreeManager::verfiyBTree ), which also checks for any structure
corructions in all btree nodes
- Added btree check to CegoDistManager::verifyTable
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Version passed concurrent load test with 10 million concurrent
operations ( random insert, update and delete on one table with 2
processes )
- Fix in CegoFileHandler::allocatePage, In case of page exceed with
appendMode = true, a second try is done with appendMode = false
- Small fix in CegoObjectManager::reorgObject, ref page id information
for append mode is set to new last page id
- Small fix in CegoSelect::filterRefs, allow expressions with constant
values ( getAttrRefList= 0 ) in aggregation
- Locking improvements for delete and update operations.
All delete operations are done now via transactions ( introduced
CegoTableManager::deleteDataTableAtomic ). This allows to use a
shared lock as the table lock for using the table
(CegoDistManager::deleteLocalDataTable and
CegoDistManager::updateLocalDataTable )
As a result, a table can be accessed in parallel during a ( long
running ) delete operation on the table( e.g. delete from table).
E.g. select count(*) from table will not block anymore, rather
indicates the remaining number of tuples in the table.
This modification is still under investigation and must verfied
( also in combination with btree/index objects on the table )
- Fix in CegoBTreeManager::deleteBTree, the relevance value was not
calculated correctly
Fix in CegoQuery, cache invalidation is done before query operations.
Otherwise, if the query is aborted, an exception is thrown and
the cache is not cleaned
- Small modifications on cgblow load simulation client
- Changed locking for delete an update operations to exclusive due
to concurrency prpblems with btree operations
- Added page locking to AVLIndexManager and AVLIndexCursor
- Improvements for locking done. Since locking is done on page level,
Insert and delete operations can be done now in parallel . For this
locking had to be introduced in CegoBtreeManager::insertBTree and
CegoBTreeManager::deleteBTree
Further performance path for grouping queries in CegoGroupSpace.
Since the current grouping key is stored now in the class variable
_groupSchema, some expensive list copy operations could be avoided.
This improves performance for about 4%.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Improvements for query execution plan retrieval
- Fix in CegoOutput for multline format
( e.g. execution plans have been cut off )
- Performance patch in CegoTableCursor::setup. The getObjectListByTable
method was called every time, when the attribute condition has been
changed. Since this operation is pretty expensive, it was useful
to cache the retrieved index/btree objet in a local class variables.
This change results in significant query speedup for joins, where
the table cursor has to be setup very often
- Small fix in CegoAction::readChain, treating backslashes for
quoteEscapeFlag=false was commented, reactivated
- Small fix in CegoMain, default for csmode changed to value "ID"
- Fix for CegoBtreeManager::insertBTree. Duplicate handling did not
work correctly, if CegoBTreeNode::addValue returns false ( node is
full ) with a duplicate detection, we also have throw exception.
Otherwise the node is split and a duplicate is added.
- Performance improvement for nested queries ( exists select,
in select). The tablecursor for the select statement is not
released and reallocated for subsequent queries ( in CegoQueryHelper,
the select handle is reset with argument false ) This has required
a different locking strategy for the cursor object in CegoDistCursor.
The cursor object has explicit to get in use, if the nextTuple
method is called the first time.
- Added online help for query cache and table cache in cgadm
- Added further performance fix for aggregation queries in CegoExpr
and CegoQueryHelper. The aggregation list for CegoExpr now is stored
in a class variable and is just evaluated the first time, it is
retrieved. In the method CegoQueryHelper::aggregateTuple, the
increment calculation of the count aggregation has been changed
from memcpy calls to direkt pointer arithmetic
- Performance fix has been done for record locking. In many cases,
row locking could be avoided, since an appropriate locking is ensured
on data page or object level. This brings another significant
performance improvement.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoBufferPool::calcHas and CegoBufferPool::calcSegement
For large file id's the integer range for the hash key might be
exceeded, which lead to wrong address calculation.
The hash key now is calculated in long based values
- Introduced server mode with same behaviour as daemon mode, but
no child process is forked and server can be terminated with Ctrl-C
- Small formatting fix in CegoOutput
- Fix in CegoDistCursor for inner and outer joins
Inner or outer joins with additional where condition might return
incorrect result set since the condition was just evaluated inside
the join.
To correct this, the WHERE condition is also checked via evalCondition.
This required also a fix in the evalCondition method.
- Added multi segment support to CegoBufferPool. Since the pool is
divided now into several memory segments, this allows larger buffer
pool configurations.
- Improvements for output formatting regarding string functions
(cegoFunction::getReturnTypeLen). Size of required column field size
is now calculated, so the outlook looks more pretty
- Added hints from David Binderman in CegoAction::backChar and
CegoBufferPage::printPage ( changed pointer comparison to avoid
-Wextra compile warnings ) Thanks !
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Decreased default cache size values to 100k ( query + table )
- Next fileId (CegoXMLSpace::nextFID ) calculation now analyses the
existing fileId and uses the lowest available. The MAXFID attribute
in the database xml file is no more used. This avoids, that file
id exceeds in case of frequent creation and removement of tablesets.
- Some format correction in CegoAdminHandler
- Increased parameter FILMNG_MAXDATAFILE to 10000 and
TABMNG_MAXTABSET to 500
- Optimization added in CegoDisCursor::distSetup(CegoAttrCond)
If the native table cursor could not set up either FULL or PART, the
table cursor retrieved all available rows from the corresponding table.
In this case, the table also can be cached to improve subsequent
queries for this table. This fix is relevant for queries like
select * from t1 where b like '%X%', since the like condition
cannot be used by the table cursor
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoSelect::nextTuple : in case of rowlimit > 0,
the cached entry was not added to query cache
- Added many columns sample to base test
( create, insert and select for table with 60 columns )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoAction::execViewCreate.
Views must be compiled successful now immediately after creation.
Otherwise, the calculated schema is not stored correctly into
dictionary.
- Fix in CegoFactor::getId.
If sysdate is used in queries, it should be resolved, otherwise
query couldbe used from cache which returns invalid result
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- In CegoQueryHelper::evalPredicate fixed termination for INSUB- and
NOTINB queries. There was no reset at the end of the evaluation, which
might lead to open table cursors. In case of procedure queries, this
might lead to locking problems.
Since the query is reset after evaluation, query caching is no more
useful. This must be treated in an advanced way.
- Further patches in CegoTableManager::deleteDataTable,
CegoFactor::evalReferences and CegoSelect::nextTuple for advanced
query cache handling. All patches have impact on queries of the form
DELETE FROM t1 WHERE a IN ( SELECT ....)
Submitted by: Björn Lemke <lemke@lemke-it.com>
- Added performance patch in CegoFieldValue.
Values with a length up to STATICFIELDBUF size ( 100 bytes ) are handled
using a static byte array. This saves expensive memory allocation
calls in most cases. First very rough performances analysis has
shown a performance improvement of about 5-10 percent
- The local copy logic was shifted from CegoSelect::nextGroupedTuple
to CegoGroupSpace::insertTuple.
Submitted by: Björn Lemke <lemke@lemke-it.com>
- Fix in CegoSelect::nextGroupedTuple.
For the 'grouping key' values, a local copy of CegoFieldValue has to be
created. Otherwise, grouping references might be invalidated, which result
in aborted grouping queries with strange error message ( No token
available ) on client side.
For local copy creation, the CegoFieldValue::getLocalCopy method has
been modified ( was not used until now ).
This effect was observed with small buffer poolsizes in
combination with grouping queries on large tables. The probability
of grouping key invalidation is higher in that case, because
buffer pool pages are reloaded more often.
Submitted by: Björn Lemke <lemke@lemke-it.com>
