Last working day at Pramati

Every start has an end...and day i.e 13th Jan 2017 is my last working day at Pramati.

I was a noob when I joined and now in a good shape :)

thanks to all with whom I worked with and learned from them...

Adding system property to point NonDefault FireFox in selenium

When we start selenium server for firefox, it would try to lauch firefox from default location. To enable us to lauch firefox from different location we need to set the system property firefoxDefaultPath with the LOCATION/firefox.exe
 To add this in ant script we need to do something like below
<property location="C:/Program Files (x86)/Mozilla Firefox-3.6.18/firefox.exe" name="browserExe"></property>
<exec executable="cmd.exe" os="${os.name}" spawn="true">
 <arg line="start">
 <arg value="java">
 <arg value="-jar">
 <arg value="${sel.server.path}${sel.server.jar}">
 <arg value="-DfirefoxDefaultPath=${browserExe}">
</exec>

JBoss startup time with Spring and Struts

After migrating to JBoss 6.1.0, Spring 3.0.6 (were are using Struts 2) our application server started slowing down on strut-up. Before the migration if I am correct it used to take about around less then a minute and after migration it was taking around 3 mins.
 Recently I got new lappy at work, with new configuration the boot up was good but not so good, now it took me around 1min 30secs to boot.
 To know the startup issue, I profile the jvm with jvisualvm and profile data showed that time was taken by spring to initialize itself. Then to make things to initialize lazily, changed the xml configuration of spring and set default-lazy-init to true for every "beans" tag.
 Now I started JBOSS expecting it to start in secs, but then it again had the same story, it took same amount of time. Then again I profiled the jvm and saw there was no difference, then to trace down I noticed that even if beans were marked as lazy there were being initialized by struts by method
 com.opensymphony.xwork2.spring.SpringObjectFactory.getClassInstance(String) [This is found in xwork jar we have 2.0.4 version]

By looking at the method it suppose to get the class instance only and not create bean, but the code does have a call like below

clazz = appContext.getBean(className).getClass();

The above code is actual culprit, to get class instance the call is creating bean and then getting the class, the code should have been as

clazz = appContext.getType(className);

As our is web application I create a copy of this class with same package name and placed it in our WEB-INF\Classes dir (as this would be first to get loaded and then *.jar) this made sure that our customized class (SpringObjectFactory) would be loaded first by jboss server and not SpringObjectFactory from xwork.jar (other way's we can do is add agents to manipulate the class method using BCEL or Javassist)

After this changes, I started JBOSS, now it takes me about 37 to 39 secs too boot (still slow but its ok for now).
Wanted to give a patch for above fix in SpringObjectFactory, but found that xwork site is in RIP state, and I could not find any other version system to give above fix.

OpenJDK 8 Build Ubuntu

Just done with OpenJDK 8 Build Ubuntu, it was easiest of all builds then done on Windows system. Had to run "configure" and it gave all the required commands to install required software for build....

Exporting and Importing Oracle Dump into XE

Today I was struggrlnig with exporting and importing Oracle dump into Oracle XE, at the end the site which came handy was stackoverflow-howto-import-an-oracle-dump-in-an-different-tablespace

Daily activities Day 1 ;)

Will try to post my daily activities technically, as it would brush up things for me once I revist what I had done and it would give me to rethink on furture of wat I should do...

So as of now I was goin through parrallel classloading, acutaly was goin through openJDK 8 mail list and saw its being drop-off from Jdk 8.

6.37 Am
Parallel classloading

8:42 Am
A nice blog Now vs. 5 Years ago

NPE issue (my stupidity)
Was working on something and when I deployed the changes and started jboss it was ending up in NPE, I had this code change made as below


Properties props = new Properties();
props.put("mail.smtp.auth", "true");
System.setProperties(props); // this is the bug....we are overriding every thing :(

And when I was starting jboss, jboss server was ending up with NPE and was not able to user.dir.

Sum of two or more independent count querries

I got a situation where I needed sum of two independent table counts, had too search and got a querry as below.


Select count1 + count2 as total from
(Select count(*) as count1 from TABLE1 an where ....),
(Select count(*) as count2 from TABLE2 atf where ....)

my current interests/readings/pending-items

I got interested in knowing assembly, so I have started reading "Assembly Language Step By Step: Programming With Linux by Jeff Duntemann".

Its too good book and as I never done assemblly this book gave me the path to go unknown world (which I should have been mastered by now, but at-least I have started on it now) and I have written my first "Hello World!" programming that prints "Hello World!" for 10 times :) ya sounds simple...but for me its a great step :)

I started looking into openJDK last year some where around Jan-11, but I am not able to give time for it and I am still struggling with debugging the code and still getting to know the code flows, its like I am still a kid wondering in the forest, I hope I will reach top of the mountain in that forest.

I got so much to learn which are pending :(.....,like still need to be good in algo's, concurrency, Lambda's in java (1.8), NIO (this I never touched, not even theory of it), NLP (using java and phyton).

I hope I will be mastering this all...one day....but ya soon...

New build system of openJDK (jdk8 master forest) rocks!

Just tried with new build system in OpenJDK (jdk8 master forest) on Windows, now there is no pain as it was with the old build. Just start the build and relax....

If any one interested to try out just check this link "http://openjdk.java.net/projects/build-infra/guide.html"

PS: if you end up with "Defs-utils.gmk: No such file or directory", just go and create a dummy file named "Defs-utils.gmk" in "jdk/makefiles/common/shared"

Lesson learnt during writing Java Agent

"Never Use Class Object reference which you want to transform in the 'agent classes' itself".

To understand above sentence, lets take an example if I want to instrument class "com.my.target.Klass1" and some where in agent jar file itself we made a reference to Klass1 object by saying
"Klass1.class"
then this class will not be called for transformation by registered ClassFileTransformer in the agent.

In my case I wanted "Klass1" name so I added code in ClassFileTransformer implementation as below:

String klassObject = Klass1.class.getName();

and when I made agent jar and wanted to use it for instrumentation, this class (Klass1.class) was no where to be seen. I struggle for hours and noticed that all other classes were called in by ClassFileTransformer implementation and only "Klass1" class was not. I had to change above code not to refer to class object directly. So I removed "Klass1.class" statement and replaced it with complete class name as below
String klassObject = "com.my.target.Klass1";

Now it worked like a charm...

Threads Class Hierarchy OpenJDK (Ref thread.hpp)

Below is taken from hotspot\src\share\vm\runtime\thread.hpp file

Class hierarchy

Thread
   |-> NamedThread
          |-> VMThread
          |-> ConcurrentGCThread
          |-> WorkerThread
                  |-> GangWorker
                  |-> GCTaskThread
   |-> JavaThread
   |-> WatcherThread

Comments from OpenJdk Source

When going through openJDK source found good comments in the source file, thought it would help me a lot for reference....so blogging it

Source: jdk7-b147 (tag)
File Name: hotspot\src\share\vm\memory\space.hpp [line number from 48 to 65]
//  A space is an abstraction for the "storage units" backing
// up the generation abstraction. It includes specific
// implementations for keeping track of free and used space,
// for iterating over objects and free blocks, etc.

// Here's the Space hierarchy:
//
// - Space-- an asbtract base class describing a heap area
// - CompactibleSpace -- a space supporting compaction
// - CompactibleFreeListSpace -- (used for CMS generation)
// - ContiguousSpace -- a compactible space in which all free space
// is contiguous
// - EdenSpace -- contiguous space used as nursery
// - ConcEdenSpace -- contiguous space with a 'soft end safe' allocation
// - OffsetTableContigSpace -- contiguous space with a block offset array
// that allows "fast" block_start calls
// - TenuredSpace -- (used for TenuredGeneration)
// - ContigPermSpace -- an offset table contiguous space for perm gen

----- END of hotspot\src\share\vm\memory\space.hpp

Hotspot Runtime Overview

OpenJDK's doc contains one very important note/document related to JVM runtime overview, this should have been read by me way back but its never late until you do it :)....so here is the link RuntimeOverview. I enjoyed it reading and came to know very key point/areas about the code in OpenJDK.

I tried with WinDbg but found it much easier with VC10 debuging executable found at this link "How to: Debug an Executable Not Part of a Visual Studio Solution".
I am still going through the OpenJDK by debuging and found few useful classes, like for example we have used few JVM options like PermSize which are defined in the file "hotspot/src/share/vm/runtime/globals.hpp", here you can find all the global JVM options defined and we have used in our day to day java application.

First step towards windows debugging for OpenJDK

Yesterday I have installed WinDbg for the purpose of debugging openJDK (mostly around b118), I haven't started using it, will update how it goes

Make downgrade from 3.81 to 3.80

I was trying to build OpenJdk-b118 (for hotspot patch), for this I had to setup few things (as I never did it at my home system), then I realized that my blog had one state for MAKE just saying "Installed make: make version 'GNU Make 3.80'", which was not so simple as is sounds.
I had to struggle for an hour or more to just downgrade the make from 3.81 to 3.80, my mistake previously that I never made a note of it how to do it (downgrading from 3.81 to 3.80), but please note that this information is not complete as well, as I don't remember exactly from where I got the make 3.80 on to my office computer.
TO downgrade, I had to have copy to files one is "make.exe" with version 3.80 and the other is "cygintl-2.dll" (as I said I don't remember how exactly I got this two files, but hope this will be helping me or some other person struggling with downgrade).

argument parsing in Hotspot

Argument parsing in hotspot is done by the class "hotspot\src\share\vm\runtime\arguments.cpp", for example argument "-Xss" would be parsed in this (will be writing more on this when I go through it thoroughly).

Mercurial tutorial

Just found a good Mercurial tutorial at http://hginit.com/00.html

Building OpenJDK 7 (openjdk-7-ea-src-b105-13_aug_2010) On Windows [Service Pack 3, Version 5.1]

Building OpenJDK 7 (openjdk-7-ea-src-b105-13_aug_2010) On Windows [Service Pack 3, Version 5.1]


Source used: openjdk-7-ea-src-b105-13_aug_2010
Local Path Source: Windows path "E:\dev\jdk7\openjdk-7-ea-src-b105-13_aug_2010" and cygwin path "/cygdrive/e/dev/jdk7/openjdk-7-ea-src-b105-13_aug_2010"
Boot strap JDK: jdk1.6.0_18
Installed the Binary Plugs: downloaded it from "http://www.java.net/download/openjdk/jdk7/promoted/b92/jdk-7-ea-plug-b92-windows-i586-06_may_2010.jar"
Installed DirectX: "DirectX 9.0 SDK Update - (Summer 2004)" as given in the "README-builds.html" file
Installed make: make version "GNU Make 3.80"
Installed VC: Microsoft Visual C++ 2010 Express


Basic environment variables
# Boot strap JDK, windows path "D:\Java\jdk1.6.0_18"
export ALT_BOOTDIR=D:/JAVA/JDK16~1.0_1
# Binary Plugin
export ALT_BINARY_PLUGS_PATH=E:/DEV/JDK7/JDK7EA~2/OPENJD~1
# DirectX path
export ALT_DXSDK_PATH=/cygdrive/e/setup/directX
# Set OPENJDK to true
export OPENJDK=true

Task 1: Run "make sanity" with out errors
------------------------------------------------
Started cygwin and exported all the ALT_ variables above and ran the sanity by "make sanity".
The first sanity make ended in below error "ERROR: FreeType version 2.3.0 or higher is required.", I tried to installed freetype for cygwin, and re-ran the sanity, but ended up with same error.

Searched and found an direction from the link belwo which says "It's trying to build a freetype check program with VS2010, and probably found a stdio.h from the CYGWIN /usr/include, not the stdio.h you want.........which means downloading freetype sources and building it, and saving those files. Yes, I know, painful. Sorry in advance.".
That means I needed to download freeType and build it localy.

http://markmail.org/thread/g6zrjtpmo4sda7aw#query:freetypecheck.c%2829%29%20%3A%20fatal%20error%20C1083%3A%20Cannot%20open%20include%20file%3A%20%27stdio.h%27%3A%20No%20such%20file%20or%20directory+page:1+mid:4bd7hun3tmbwmjql+state:results

So downloaded FreeType version 2.4.4 (freetype-2.4.4) and tried to build using make in cygwin, but ended up with below error
make: ver: Command not found
make: type: Command not found
make: *** [dos_setup] Error 127
Couldn't find a solution (tried a lot to search on web), then using "VC++ 2010 Exp" I build the freetype as explained below

1. Double clicked the file "freetype-2.4.4\builds\win32\visualc\freetype.dsp"
2. Clicked next in "Conversion wizard", if needed choose backup before convertion, and click next and finish.
3. In "Solution Explorer", right clicked and choosen "Configuration Properties"->"General" in left panel, and in the right panel changed "Configuration Type" to "Dynamic Library (.dll)"
4. And made a build (F7), this will create freetype.dll in "freetype-2.4.4\objs\debug".

freetype.lib too is need for the build, we could use VC 2010 for this, but I couldn't genereated the lib file, instead I build freetype.lib using "ftjam-2.5.2-win32"
1. download, extact and set ftjam's bin in path.
2. Open "Visual Studio Command Prompt 2010" found in start up menu.
3. Changed directory to the root of the freetype-2.4.4, and type jam.
4. freetype.lib is created in "freetype-2.4.4\objs"

Copy this freetype.dll and freetype.lib in a directory, in my case I created "lib" directory in "E:\dev\jdk7\freetype-2.4.4_V2\freetype-2.4.4\lib" folder.

Now export the ALT_FREETYPE_LIB_PATH in cygwin environment
export ALT_FREETYPE_LIB_PATH=E:/DEV/JDK7/FREETY~2.4_V/FREETY~1.4/LIB

Note: To get the path as above "E:/DEV/JDK7/FREETY~2.4_V/FREETY~1.4/LIB", the simple solution is, first using cmd go to the path, in my case I typed
"cd cd E:\dev\jdk7\freetype-2.4.4_V2\freetype-2.4.4\lib" and then type "command" we get "E:\DEV\JDK7\FREETY~2.4_V\FREETY~1.4\LIB", now replace '\' with '/' and we get the path.

Now set the ALT_FREETYPE_HEADERS_PATH
export ALT_FREETYPE_HEADERS_PATH=E:/DEV/JDK7/FREETY~2.4_V/FREETY~1.4/INCLUDE

Here I was able to successfully run the sanity check.


Task 2: Building OpenJdk
------------------------------------------------
After sucessfull sanity check, we can go ahead and build the system. Just type make, and I ended with below error

D:/PROGRA~1/MICROS~1.0/VC/Bin/rc: No such file or directory

RC.exe is no more available in VC++ 2010, instead its available in Microsoft SDK, my system contains Microsoft SDK at location "C:\Program Files\Microsoft SDKs"
Now export variable MSSDKTOOLS_PATH as below which should point to BIN directory of Microsoft SDK

export MSSDKTOOLS_PATH=C:/PROGRA~1/MI2578~1/WINDOWS/V7.0A/BIN

And change the content of the file Compiler-msvc.gmk as below located at "openjdk-7-ea-src-b105-13_aug_2010\jdk\make\common\shared"

"RC = $(MSDEVTOOLS_PATH)rc" to "RC = $(MSSDKTOOLS_PATH)rc"
"RSC = $(MSDEVTOOLS_PATH)rc" to "RSC = $(MSSDKTOOLS_PATH)rc"

its enough to change MTL and MT for only VC 10, ie in the if block of COMPILER_NAME as Visual Studio 10
"MTL = $(MSDEVTOOLS_PATH)/midl.exe" to "MTL = $(MSSDKTOOLS_PATH)/midl.exe"
"MT = $(MSDEVTOOLS_PATH)mt" to "MT = $(MSSDKTOOLS_PATH)mt"

Here basically we are pointing RC, midl and mt to be picked from microsoft sdk rather from Visual Studio 10

That should build OpenJDK.

java.sql.SQLException: ORA-06502: PL/SQL: numeric or value error: character string buffer too small

I got an issue where I had to fix the above problem, what we found was we had a VIEW where we were converting a CLOB to String (i.e. VARCHAR2) using DBMS_LOB.SUBSTR, there was no problem till now and we were using this view from long long time, the above situation came when one of the rows in the CLOB column had double-byte characters and it extended VARCHAR2 length i.e. 4000+, now when we were using this VIEW it was failing saying "ORA-06502" error.

DBMS_LOB.SUBSTR we were using as "DBMS_LOB.SUBSTR(COMMENTS, DBMS_LOB.GETLENGTH(COMMENTS))", and we changed it to DBMS_LOB.SUBSTR(COMMENTS, 1000), as the double-byte characters would take 4 bytes.

I think the above fix is a quick one, and the issue with this would be for normal chars we would be getting max 1000 chars from the CLOB where as in that CLOB it might be more then that.

Will post when we I have a fix for it :)