blog. aregger.io

A blog about Oracle, SQL and Java

twitter github

Loading data into index organized tables

19 Jun 2016

Most of the time you don’t have full control of how your data gets loaded into the database, because it is often imposed by business rules. But in the case you have full control, you should take advantage of it.

I recently came across an index organized table which was kind of a denormalized form of other tables and which was truncated and reloaded every day before some heavy batch jobs queried it.

The data was loaded into the IOT with an insert as select statement. Because the data returned by the select was not ordered (or at least not ordered after the primary key columns of the IOT), the insert caused a lot of 50/50 block splits on the IOT.

I added a simple order by clause to the statement with the effect that only 90/10 block splits were done afterwards. This also resulted in a 50% smaller index size.

Here’s a small test case for demonstration:

Create a heap table which is later used to load data into the IOT

The c1 column will be the pk column of the later created IOT. I used a random function so the values for this column are not ordered in any way.

create table t_heap (c1 number, c2 varchar2(100));

INSERT INTO t_heap
SELECT dbms_random.random, lpad('#',100,'#')
FROM dual CONNECT BY rownum <= 100000;

Remove any duplicate values for c1 as this would lead to unique constraint violations later.

declare
 lastValue t_heap.c1%type;
 cursor c_heap is select * from t_heap order by c1 for update;
begin
  for rec in c_heap loop
    if rec.c1 = lastValue then
      delete from t_heap where current of c_heap;
    end if;
  lastValue := rec.c1;
  end loop;
end;
/

commit;

Create the IOT

create table t_iot (c1 number, c2 varchar2(100),
 constraint pk_iot primary key (c1))
 organization index;

Check the index split statistics before inserting

select n.name,s.value from v$mystat s
join v$statname n on (s.statistic# = n.statistic#)
where n.name in ('leaf node splits', 'leaf node 90-10 splits');

NAME                                                                  VALUE
---------------------------------------------------------------- ----------
leaf node splits                                                          0
leaf node 90-10 splits                                                    0

Insert without order by clause

insert into t_iot select * from t_heap;


NAME                                                                  VALUE
---------------------------------------------------------------- ----------
leaf node splits                                                       2908
leaf node 90-10 splits                                                    0

select bytes/1024/1024 from dba_segments where segment_name='PK_IOT';

BYTES/1024/1024
---------------
             23

About ~3k block splits were done, all of them 50/50. The size of the index is ~23 MB.

Now insert with order by clause

(Open a new database session, so we have “fresh” counters in v$mystat)

truncate table t_iot;
insert into t_iot select * from t_heap order by c1;

NAME                                                                  VALUE
---------------------------------------------------------------- ----------
leaf node splits                                                       1458
leaf node 90-10 splits                                                 1458

select bytes/1024/1024 from dba_segments where segment_name='PK_IOT';

BYTES/1024/1024
---------------
             12

The amount of block splits is cut in half and because of the order by clause, only 90/10 block splits were done. The index size is also considerably smaller (12 MB compared to 23 MB).