Monitoring ZFS Statistic on the system loaded with Oracle database

Nothing happens without a reason. My previous post about Monitoring ZFS Statistic is not exception. This week I had T5240 (128 threads, 64GB memory, Solaris 10 10/08 aka U6) with 6140 attached and not loaded with other my projects. So I decided to load the boxes with Oracle on ZFS (binaries, datafiles, logs) and run some tests. Not a benchmarking. Just to check if this can live and breathe. I also wanted to check the battle for the memory between Oracle SGA (set to 32GB), 3800 processes connecting to database (taking approx. 5MB not shared memory each) and ZFS.

On storage I have made single 1TB RAID1 LUN and put ZFS on it. No data/logs of ZFS separation. Keeping it simple.

root@rac05#zpool list
NAME     SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
oracle  1016G   243G   773G    23%  ONLINE  -
 

Arcstat script was modified a little further (lines 82, 177, 229 and 230) so I don't have too many zeros in statistics by going up from bytes to MB. Download new arcstat script and description file for dimstat.

 81 my $version = "0.1";
 82 my $mega = 1024 * 1024;
 83 my $cmd = "Usage: arcstat.pl [-hvx] [-f fields] [-o file] [interval [count]]\n";
...
171 sub prettynum {
172     my @suffix=(' ','K', 'M', 'G', 'T', 'P', 'E', 'Z');
173     my $num = $_[1] || 0;
174     my $sz = $_[0];
175     my $index = 0;
176     return sprintf("%s", $num) if not $num =~ /^[0-9\.]+$/;
177     return sprintf("*%d", $sz, $num);
178     while ($num > 1000 and $index < 8) {
179         $num = $num/1000;
180         $index++;
181     }
182     return sprintf("%*d", $sz, $num) if ($index == 0);
183     return sprintf("%*d%s", $sz - 1, $num,$suffix[$index]);
184 }
...
200 sub calculate {
...
229     $v{"arcsz"} = $cur{"size"}/$mega;
230     $v{"c"} = $cur{"c"}/$mega;
...
238 }

On the picture you may see results of monitoring:

At 1:01 test was started. It took 1 hour until all processes become ready (1 process was started each 1 second). Then Oracle database was busy between 2:04 and 2:53.

As you can see ZFS is taking more memory if it can, but it stays "polite" by freeing this memory for other processes when it needed.

I will be glad to hear from you if you have any thoughts and experience running Oracle Database on ZFS.

Thanks to Valery for his Perl Syntax highlighting plugin for Netbeans.