tag:blogger.com,1999:blog-86688538187839198602024-03-13T13:00:27.855-07:00CALL GLOP()Andy Malakov software blogAnonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comBlogger32125tag:blogger.com,1999:blog-8668853818783919860.post-62041186315021687282016-11-30T18:18:00.001-08:002016-12-01T20:52:59.226-08:00Built a model of Mendocino Motor<div dir="ltr" style="text-align: left;" trbidi="on">
<iframe allowfullscreen="" frameborder="0" height="270" src="https://www.youtube.com/embed/xzqMt9PdzSI" width="480"></iframe>
<br/>(<a href="https://en.wikipedia.org/wiki/Mendocino_motor">wikipedia</a>)</div>
Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-24925693108967772962015-03-04T18:30:00.000-08:002015-03-04T18:30:15.354-08:00Connecting two CentOS computers using cheap Infiniband<p>This is a continuation of the <a href="http://andy-malakov.blogspot.com/2015/03/connecting-two-windows-7-computers-with.html">previous</a> post. This time I wanted to test direct Infiniband connection on Linux.
<p>Setup is the same:
<ul>
<li>Two retired developer's desktops (built in 2008): AMD Opteron 2216 @2.4MHz, 8G DDR2.
<li>A pair of Mellanox Infinihost III adapter MHGA28-XTC
<li>CentOS Linux 6.3 (Minimal Install in my case)
<li>MLNX_OFED_LINUX-1.5.3-4.0.42-rhel6.3-x86_64.iso OFED driver (Still available on mellanox.com)
</ul>
<p>Linux setup is pretty straightforward but in my opinion more involved than on Windows. Main problem was old age of these cars. In order to avoid rebuilding OFED drivers for these cards I used old version of CentOS (6.3). I've tried 2.x version but got MFE_OLD_DEVICE_TYPE error. Besides I wanted to test SDP in Java 7 and this protocol seems to be no longer available in OFED 2.x +.
<p><b>Bottom line:</b> for these old Infinihost III-family cards use older OFED driver (1.5.3). If you don't want to rebuild the driver, use Linux distro/version specified by the driver (there are quite a few).
<p>I found that the following two resources most useful for this project:
<a href="http://www.softpanorama.org/HPC/IB/installing_mellanox_driver_for_rhel65.shtml">A</a> and
<a href="http://pkg-ofed.alioth.debian.org/howto/infiniband-howto-5.html">B</a>. There is no reason to repeat these steps here. Connection verification and testing using OFED utilities is similar to Windows version.
<p>Configuring simple Java Socket application to use SDP worked like a charm. See <a href="http://docs.oracle.com/javase/tutorial/sdp/">Oracle's tutorial</a>.
Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-10310406489278006862015-03-04T18:06:00.001-08:002015-03-04T18:06:17.709-08:00Connecting two Windows 7 computers with low-cost Infiniband<p>Previous generation Infiniband cards are selling for a fraction of original price on eBay. Developers are buying these setups to test/learn this technology. I've followed this path and posting my notes here. The setup wasn't easy and I had to collect information from various sources.
<h2>Hardware</h2>
<ul>
<li>A pair of Mellanox Infinihost III adapter MHGA28-XTC (<a href="http://www.ebay.com/sch/i.html?&_nkw=MHGA28-XTC">$36</a>)</li>.
<li>A pair of Molex 4X Infiniband copper cables (<a href="http://www.ebay.com/sch/i.html?&_nkw=4X%20InfiniBand%20cable">$12.5</a>)</li>
</ul>
<p>Total price tag was $97 (including shipping).
<p>These are old-generation Dual-Port InfiniBand adapter cards that fit into PCI Express x8 slots. Each card has two 20Gb/s ports.
I used two retired supermicro desktops (circa 2008) that fit these cards by age. Each computer is running Windows 7 (x64). [Next post will explore the same hardware setup on Linux].
<p>Infinihost allows direct connection between two computers (in point-to-point setup there is no need for Infiniband switch).
<h2>BIOS Update</h2>
<p>Multiple sources recommend upgrading card's firmware before trying them with Windows.
<p>There are several revisions of MHGA28-XTC cards, mine was A3 (check the sticker attached to the back of each card). Firmware can be downloaded from Mellanox <a href="http://www.mellanox.com/page/firmware_table_IH3Ex">here</a>.
<p>To upgrade firmware and basic status testing Mellanox provides MFT utilities set. In my case the latest MFT version 3.8 refused to work with these cards claiming they are no longer supported. <a href="https://community.mellanox.com/thread/1212">Luckily</a> MFT version 2.7.2 is still <a href="http://old.mellanox.com/content/pages.php?pg=management_tools&menu_section=34">available</a> and works with these old Infinihost-family cards:
<pre>
C:\Program Files\Mellanox\WinMFT>mst status
MST devices:
------------
mt25218_pciconf0
mt25218_pci_cr0
C:\Program Files\Mellanox\WinMFT>mlxburn -dev mt25218_pci_cr0 -image fw-25218-5_3_000-MHGA28-XTC_A3.bin
Current FW version on flash: 5.2.916
New FW version: 5.3.0
Read and verify Invariant Sector - OK
Read and verify PPS/SPS on flash - OK
Burning second FW image without signatures - OK
Restoring second signature - OK
-I- Image burn completed successfully.
</pre>
<h2>Windows Driver</h2>
<p>Initially these cards showed up as "Infiniband controller" in Windows Device Manager:
<div class="separator" style="clear: both; text-align: left;">
<img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDdAA9vU-p8czaTRGbRmYk23UiX_zgHve-fE4Uebx96R1g1RAwerARKx59zS59-0vxzEVjzlIfyvMtCQ6ieQ3xSrNqRz9o4_IdmIhkaeOVrglWzN0xBtgUERjIn6UUjizAyRgSv8E0mNQ/s1600/no-driver.png" />
</div>
<p>Driver for these cards are available from Mellanox and OpenFabrics.org (OFED). I believe both of these sources actually provide the same driver maintained by OFED (sponsored by Mellanox).
<p>Here I had the same story - the latest OFED driver version (3.2) simply didn't work with these cards. Setup ended with "Possible NetworkDirect startup failure" warning. The installed driver would identify the card properly but yellow triangle said that device was disabled due to errors. Windows event log showed that some driver components failed to initialize.
<div class="separator" style="clear: both; text-align: left;">
<img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPKYd6iWOol0r-19sWP5A6pVULXQ_hYwdsXEo5tRX373XH-GCj3vO-fS_O0NIQCtUUWRcwzeJCWxz0bZoiFDxNBYjjNbqpfh9GLQAfk6tlLCBE-pEtKT-jSeznhWlvhl9TPGDL-kzP_kw/s1600/driver-with-problem.png" />
</div>
<p>After some trial and error I found that OFED driver version 2.3 was what I needed. I can be downloaded from OpenFabrics <a href="https://www.openfabrics.org/downloads/Windows/previous_releases/v2.3/Win7-Svr_2008_R2-HPC_Edition/">archive</a>.
<p>As you can see below, in addition to Infiniband card Device Managers showed that I got two OpenFabrics IPoIB Adapters (since each card has two ports):
<div class="separator" style="clear: both; text-align: left;">
<img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrZfGymIegdtQ6rDbcEN9BrgrAbmPN_ZP68Y51ZOaKv1GR5X9uMZkjQA3mhHFyqNJkAcf0C_fLv3X62mBQe-NKe7DRDrUA7_Y8z9hy7hgTgpd1RUYzoDlZGBRcLiTTyJfWOoLvNDtA8es/s1600/OFED-drivers-success2.3.png" />
</div>
<h2>Configuration</h2>
<p>I repeating above steps on both computers and connected cards with cables.
<p>OFED software comes with set of utilities, one of which (IBSTAT) can be used to check connectivity status:
<pre>
C:\Windows\system32>ibstat
CA 'ibv_device0'
CA type:
Number of ports: 2
Firmware version: 0x500030000
Hardware version: 0x20
Node GUID: 0x0002c9020023c250
System image GUID: 0x0002c9020023c253
Port 1:
State: Initializing
Physical state: LinkUp
Rate: 20
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x90580000
Port GUID: 0x0002c9020023c251
Link layer: IB
Port 2:
State: Initializing
Physical state: LinkUp
Rate: 10
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x90580000
Port GUID: 0x0002c9020023c252
Link layer: IB
</pre>
<h3>Subnet Manager</h3>
<p>When these cards are connected directly we need to launch Infiniband Subnetwork Manager (opensm). OFED installs it as Windows Service (disabled by default). In my case I launched <em>opensm</em> from command line. You need to run this service on both computers.
<pre>
C:\Windows\system32>opensm
-------------------------------------------------
OpenSM 3.3.6 UMAD
Command Line Arguments:
Log File: %TEMP%\osm.log
-------------------------------------------------
OpenSM 3.3.6 UMAD
Entering DISCOVERING state
Using default GUID 0x2c9020023c251
Entering MASTER state
SUBNET UP
Entering STANDBY state
</pre>
<p>Each service can be configured to serve both ports (enter GUID of each port GUIDs into opensm configuration file).
<p>After this step Windows should show your network status as connected:
<div class="separator" style="clear: both; text-align: left;">
<img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKHqkp42k40LVgoTutanqv3nJ9gyVlpzaQ6Y6r5AtRx4A06gCO8PnCjrmBmKznL3vSzFQ1fgu_gvYla586FaIAfx8QPCkHornFm6oU0cCvJvO01i14ANGWAm8NXDbrcfrnRUJzUWcFbGI/s1600/OFED-networks-connected.PNG" />
</div>
If you plan to keep this setup running, OpenSM can be launched automatically as Windows Service (disabled by default).
<h3>Connectivity test</h3>
<p>OFED has special ping utility that can be used for quick test.
<p>Computer 1 (Here we print GUIDs of each port and launch ping server):
<pre>
C:\Windows\system32>ibstat -p
0x0002c90200231745
0x0002c90200231746
C:\Windows\system32>ibping -S
</pre>
<p>Computer 2 (here we use GUID of the first computer's port):
<pre>
C:\Windows\system32>ibping -G 0x0002c90200231745
Pong from ?hostname?.?domainname? (Lid 2): time 0.230 ms
Pong from ?hostname?.?domainname? (Lid 2): time 0.160 ms
Pong from ?hostname?.?domainname? (Lid 2): time 0.231 ms
Pong from ?hostname?.?domainname? (Lid 2): time 0.159 ms
Pong from ?hostname?.?domainname? (Lid 2): time 0.163 ms
Pong from ?hostname?.?domainname? (Lid 2): time 0.174 ms
</pre>
(Nevermind weird host name).
<h2>Latency test</h2>
Computer 1 (Launching test server):
<pre>
C:\Windows\system32>ib_send_lat -a -c RC
</pre>
Computer 2 (test client):
<pre>
C:\Windows\system32>ib_send_lat -a -c RC oldfaithful
------------------------------------------------------------------
Send Latency Test
Inline data is used up to 400 bytes message
Connection type : RC
test
local address: LID 0x100, QPN 0x6040200, PSN 0x265a0000, RKey 0x2c0010 VAddr 0x00000001170040
remote address: LID 0x200, QPN 0x6040600, PSN 0xb64e0000, RKey 0x2c0030 VAddr 0x00000000fc0040
Mtu : 2048
------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec]
2 1000 4.10 2295.99 4.27
4 1000 3.75 1305.61 4.27
8 1000 3.75 256.86 3.93
16 1000 3.75 266.24 3.93
32 1000 4.44 1021.62 4.61
64 1000 4.44 329.22 4.61
128 1000 4.61 303.79 4.78
256 1000 4.95 529.41 5.12
512 1000 5.46 300.89 5.63
1024 1000 6.83 309.25 7.00
2048 1000 9.22 327.17 9.39
4096 1000 11.61 280.92 11.78
8192 1000 16.90 306.86 17.07
16384 1000 27.65 329.56 27.82
...
</pre>
This hardware is 9 years old, so numbers are sub-optimal. Still much better than TCP even without any special tuning.
<h2>What's next?</h2>
<p>We got low-latency 20Gb/s connection between two Windows 7 machines using a pair of cheap Infiniband adapters.
<p>In theory this setup can be used for ultra-fast file sharing etc. My primary interest was getting my hands on Infiniband and OFED stack (and ultimately using it from Java). Unfortunately Socket Direct Protocol (SDP) available in Java since version 7 is a) deprecated in the latest version of OFED and b) seems to be unsupported by Java on Windows anyway. There are various libraries that provide RDMA to Java using JNI wrappers.
Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-85088508323495544662011-12-05T17:44:00.000-08:002011-12-05T17:45:50.883-08:00My CPU collection<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9ocAFhGSJ4QKVj8UIM1XjzoUaFWEPXuOtosJe1HU3SYHHJDeJcVevMqeNxyWjkO18S3Fx2T3S4dR-Dq9nyvb5t1Myxvs3XEVJM4Hv8S_Z0cDBsUbU4FYHJyYYUBjeN0rJFP1zY-NOQ1E/s1600/IMG_1386.JPG"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 287px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9ocAFhGSJ4QKVj8UIM1XjzoUaFWEPXuOtosJe1HU3SYHHJDeJcVevMqeNxyWjkO18S3Fx2T3S4dR-Dq9nyvb5t1Myxvs3XEVJM4Hv8S_Z0cDBsUbU4FYHJyYYUBjeN0rJFP1zY-NOQ1E/s400/IMG_1386.JPG" border="0" alt=""id="BLOGGER_PHOTO_ID_5682825505368179666" /></a>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-25507109372362561652010-10-11T14:50:00.000-07:002010-10-11T14:51:12.014-07:00ShouldNotReachHere#<br /># A fatal error has been detected by the Java Runtime Environment:<br />#<br /># Internal Error (classFileParser.cpp:3161), pid=3136, tid=4676<br /># Error: ShouldNotReachHere()<br />#Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-2749794533908170122010-10-03T22:17:00.000-07:002010-10-10T21:26:15.595-07:00My first Android app<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuHzoe2HcjGM6k7712ODAWqKH514AJgbd03L2eeOcHEUXdJoHPqS0VYyu8ZOx3AL6KgRc9a3-IcjyCKrh8kWy9qxHGdtJ41D7oHh5vWDLVYo_jsEA7oO3zVRAwnszvCjt1XmWMWgCvG7M/s1600/MayaCalendar.jpg"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 276px; height: 320px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuHzoe2HcjGM6k7712ODAWqKH514AJgbd03L2eeOcHEUXdJoHPqS0VYyu8ZOx3AL6KgRc9a3-IcjyCKrh8kWy9qxHGdtJ41D7oHh5vWDLVYo_jsEA7oO3zVRAwnszvCjt1XmWMWgCvG7M/s320/MayaCalendar.jpg" alt="" id="BLOGGER_PHOTO_ID_5526639980424558594" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU5_nIVBC8nuOtk37cQnqvX7spP3Enihk8zw0Ft7aq2ookIMPaKubOO0eaGn9oJ5eljbAf6NlE4jW8mG3ey-gFEQbd836KkIjpY6dMu_PYxGghE2WiOgnUmYMkov1VNYFGAhWNFmsfZgg/s1600/maya-calendar-b1.jpg"><br /></a><br /><br /><p>I created my first Android app. It converts a date in Gregorian Calendar into Maya "Long Count" calendar. Contrary to popular view Mayan calendar doesn't end on December 21st, 2012. You can test this in my app :-).<br /><br /></p><p>Sources can be found <a href="http://code.google.com/p/maya-calendar-android/">here</a>.</p>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-84216894368758406472010-06-06T19:12:00.000-07:002010-06-07T13:57:57.212-07:00Alternative to Thread.sleep()Follow-up to my <a href="http://andy-malakov.blogspot.com/2010/05/measuring-nanoseconds-in-java-windows.html">previous post</a>. Here is an alternative to Thread.sleep() that uses spin-yield:<br /><pre><span class="s0"><a name="l11"><br /></a><a name="l12"> </a></span><span class="s1"><a name="l12">private static final long </a></span><span class="s0"><a name="l12">SLEEP_PRECISION = TimeUnit.MILLISECONDS.toNanos(</a></span><span class="s2"><a name="l12">2</a></span><span class="s0"><a name="l12">); </a></span><span class="s3"><a name="l12">//TODO: Determine for current machine</a></span><span class="s0"><a name="l12"><br /></a><a name="l13"><br /></a><a name="l14"> </a></span><span class="s1"><a name="l14">public static void </a></span><span class="s0"><a name="l14">sleepNanos (</a></span><span class="s1"><a name="l14">long </a></span><span class="s0"><a name="l14">nanoDuration) </a></span><span class="s1"><a name="l14">throws </a></span><span class="s0"><a name="l14">InterruptedException {<br /></a><a name="l15"> </a></span><span class="s1"><a name="l15">final long </a></span><span class="s0"><a name="l15">end = System.nanoTime() + nanoDuration;<br /><br /></a><a name="l16"><br /></a><a name="l17"> </a></span><span class="s1"><a name="l17">long </a></span><span class="s0"><a name="l17">timeLeft = nanoDuration;<br /></a><a name="l18"> </a></span><span class="s1"><a name="l18">do </a></span><span class="s0"><a name="l18">{<br /></a><a name="l19"> </a></span><span class="s1"><a name="l19">if </a></span><span class="s0"><a name="l19">(timeLeft > SLEEP_PRECISION)<br /></a><a name="l20"> Thread.sleep (</a></span><span class="s2"><a name="l20">1</a></span><span class="s0"><a name="l20">);<br /></a><a name="l21"> </a></span><span class="s1"><a name="l21">else</a></span><span class="s0"><a name="l21"><br /><br /></a><a name="l22"> Thread.sleep (</a></span><span class="s2"><a name="l22">0</a></span><span class="s0"><a name="l22">); </a></span><span class="s3"><a name="l22">// Thread.yield();</a></span><span class="s0"><a name="l22"><br /></a><a name="l23"> timeLeft = end - System.nanoTime();<br /></a><a name="l24"><br /></a><a name="l25"> </a></span><span class="s1"><a name="l25">if </a></span><span class="s0"><a name="l25">(Thread.interrupted())<br /></a><a name="l26"> </a></span><span class="s1"><a name="l26">throw new </a></span><span class="s0"><a name="l26">InterruptedException ();<br /></a><a name="l27"><br /></a><a name="l28"> } </a></span><span class="s1"><a name="l28">while </a></span><span class="s0"><a name="l28">(timeLeft > </a></span><span class="s2"><a name="l28">0</a></span><span class="s0"><a name="l28">);<br /><br /></a><a name="l29"> }</a></span></pre><a name="l29"><br /></a><br /><br /><h2>Test</h2><br />I run 4 threads requesting 5 millisecond sleep 1000 times each (on my Dual-Core CPU). The first chart shows <tt>sleepNanos(TimeUnit.MILLISECONDS(5).toNanos())</tt>:<br /><br /><img style="display:block; margin:0px auto 10px; text-align:left;cursor:pointer; cursor:hand;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5aK1rir_O2tp3klcxQKDeX70kqyx9B3abbqProvua9cW0BNlNd1eBYIlZBOC1ryL0lFGjJJ7bbPqxviNSSItA2qbcQMRt_s-MbgjvimJZqBkkH9DjWvaqXs5z9VgZAuUAO0jOJt8TfZ4/s1600/SleepStatsNanos.jpg" border="0" alt="Actual sleep time of sleepNanos(5000000)"/><br /><br />The second chart shows <tt>Thread.sleep(5)</tt>:<br /><br /><img style="display:block; margin:0px auto 10px; text-align:left;cursor:pointer; cursor:hand" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRi6tBK1Tp1HDhdI-GBvbP66aRdRZFbShKGSt0qFrIgPIAH5j5yq83NSk5ijVu0oJ40V2i4jOhN2bC1VeaKv-fVDql6rcyuQo8cAdAnPowQYE_8jbGFkfirjyz30mx2zjwev0dbcsn9VY/s1600/SleepStatsStd.jpg" border="0" alt="Actual sleep time of Thread.sleep(5)" /><br /><br />As you can see, sleepNanos() is much more precise. I found that this approach was originally used by <a href="http://www.geisswerks.com/ryan/FAQS/timing.html">Ryan Geiss for WinAmp visualization plugin</a>.<br /><br /><h2>UPDATE</h2><br /><br />Even better precision can be achieved if you are willing to consume more CPU power doing Spin-Wait for the last part of the wait:<br /><br /><img style="display:block; margin:0px auto 10px; text-align:left;cursor:pointer; cursor:hand;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAtA_-JdkOJF9325B9KKxC1cz7Kq861CxQAwBPy_CDWGdPa2uAgGfI7JxCDKhiNA1RkukKVN-0iSiLmoxI5C58eNSiA2KBqdoIHKcmiuI2YdmwDedsWHSls51CnvuP0LORjwJrdS07Cc4/s1600/sleepSpinNanos.JPG" border="0" alt=""id="BLOGGER_PHOTO_ID_5480136686943990210" /><br /><br /><br /><pre><br /><span class="s0"> <br /><a name="l14"> <br /></a><a name="l15"> </span><span class="s1">public static void </span><span class="s0">sleepNanos (</span><span class="s1">long </span><span class="s0">nanoDuration) </span><span class="s1">throws </span><span class="s0">InterruptedException { <br /></a><a name="l16"> </span><span class="s1">final long </span><span class="s0">end = System.nanoTime() + nanoDuration; <br /></a><a name="l17"> </span><span class="s1">long </span><span class="s0">timeLeft = nanoDuration; <br /></a><a name="l18"> </span><span class="s1">do </span><span class="s0">{ <br /><br /></a><a name="l19"> </span><span class="s1">if </span><span class="s0">(timeLeft > SLEEP_PRECISION) <br /></a><a name="l20"> Thread.sleep (</span><span class="s2">1</span><span class="s0">); <br /></a><a name="l21"> </span><span class="s1">else</span><span class="s0"> <br /></a><a name="l22"> </span><span class="s1">if </span><span class="s0">(timeLeft > SPIN_YIELD_PRECISION) <br /><br /></a><a name="l23"> Thread.sleep(</span><span class="s2">0</span><span class="s0">); <br /></a><a name="l24"> <br /></a><a name="l25"> timeLeft = end - System.nanoTime(); <br /></a><a name="l26"> <br /></a><a name="l27"> </span><span class="s1">if </span><span class="s0">(Thread.interrupted()) <br /></a><a name="l28"> </span><span class="s1">throw new </span><span class="s0">InterruptedException (); <br /></a><a name="l29"> <br /></a><a name="l30"> } </span><span class="s1">while </span><span class="s0">(timeLeft > </span><span class="s2">0</span><span class="s0">); <br /><br /></a><a name="l31"></a> }</span></pre>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-34452141180956884272010-05-13T10:35:00.001-07:002010-05-17T19:42:27.051-07:00Measuring nanoseconds in Java / Windows<p>I was under false impression that interval timers in Java allow sub-millisecond precision on Windows. I knew that Thread.sleep(millis, nanos) internally uses milliseconds, but for some reason I thought that methods like <tt>LockSupport.parkNanos()</tt> method provide precise waits. Well, I was wrong. The smallest delay this method can realize is approximately 1.95 milliseconds on my Windows PC.</p><br /><br /><p>Back in 2006 David Holmes <a href="http://blogs.sun.com/dholmes/entry/inside_the_hotspot_vm_clocks">explained</a> that Java timer intervals are based on <tt>waitForMultipleObjects()</tt> Windows API (which uses dwMilliseconds). They still are.</p><br /><br /><p>One simple (but not universal) workaround is a "spin-sleep":<br /><pre><br /> private static void sleepNanos (long nanoDelay) {<br /> final long end = System.nanoTime() + nanoDelay;<br /> do {<br /> Thread.yield(); // Thread.sleep (0);<br /> } while (System.nanoTime() < end);<br /> }<br /></pre><br /><br />When running standalone this method consumes all free resources of single CPU core, but it will share with other threads that may be running. For large durations it can be enhanced to use Thread.sleep() for bulk of the waiting.</p><br /><br />P.S. Difference between Thread.yield() and Thread.sleep( 0 ) is explained <a href='http://www.javamex.com/tutorials/threads/yield.shtml'>here</a>.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-62558630244762653642009-12-11T14:40:00.000-08:002009-12-11T15:03:04.150-08:00Old trapI was puzzled why one of my objects had incorrect toString()<br /><br /><pre><span class="s0">class </span><span class="s1">Foo { <br /><a name="l5"> </a></span><span class="s0"><a name="l5">private long </a></span><span class="s1"><a name="l5">id; <br /></a><a name="l6"> </a></span><span class="s0"><a name="l6">private </a></span><span class="s1"><a name="l6">String name; <br /></a><a name="l7"> <br /></a><a name="l8"> Foo(</a></span><span class="s0"><a name="l8">long </a></span><span class="s1"><a name="l8">id, String name) { <br /></a><a name="l9"> </a></span><span class="s0"><a name="l9">this</a></span><span class="s1"><a name="l9">.id = id; <br /></a><a name="l10"> </a></span><span class="s0"><a name="l10">this</a></span><span class="s1"><a name="l10">.name = name; <br /></a><a name="l11"> } <br /></a><a name="l12"> <br /></a><a name="l13"> </a></span><span class="s0"><a name="l13">public </a></span><span class="s1"><a name="l13">String toString () { <br /></a><a name="l14"> </a></span><span class="s0"><a name="l14">return </a></span><span class="s1"><a name="l14">id + </a></span><span class="s2"><a name="l14">' ' </a></span><span class="s1"><a name="l14">+ name; <br /></a><a name="l15"> } <br /></a><a name="l16">}</a></span></pre><br /><br />The statement <tt>new Foo(123, "Hello")</tt> prints something like "155Hello" instead of "123 Hello". Of course the reason is pretty simple:<br /><br /><pre><span class="s0"> <br /><a name="l18"> </a><a name="l19"> </a></span><span class="s1"><a name="l19">public </a></span><span class="s0"><a name="l19">String toString2() { <br /></a><a name="l20"> </a></span><span class="s1"><a name="l20">return new </a></span><span class="s0"><a name="l20">StringBuilder().append(id + </a></span><span class="s2"><a name="l20">32L</a></span><span class="s0"><a name="l20">).append(name.toString()).toString(); <br /></a><a name="l21"> }</a></span></pre>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-89040185039534544612009-11-11T08:45:00.000-08:002009-11-11T16:23:30.996-08:00Stack allocation in Java is still a myth<p>There were rumors that Mustang will have <a href='http://www.ibm.com/developerworks/java/library/j-jtp09275.html'>on stack allocation</a> as a part of hot spot optimization.<br /><br /><p>Four+ years later this :<br /><pre class="csharpcode"><br /><span class="kwrd">public</span> <span class="kwrd">long</span> encode (String input) {<br /> final <span class="kwrd">byte</span> [] buffer = <span class="kwrd">new</span> <span class="kwrd">byte</span> [8]; <br /><br /> .. encode input into buffer <br /><br /> .. convert buffer into LONG <br /><br /> <span class="kwrd">return</span> result;<br />}</pre><br /><br />I was hoping JVM will allocate <tt>buffer</tt> on stack based on the fact that it does not escape from this method. Running this test 10M times with <tt>-verbosegc</tt> shows extensive GC work (1.6.0_16-b01 64 bit server JVM with -XX:+DoEscapeAnalysis option).<br /></p><br /><br />On the positive side. GC is very fast. Consider three functions:<br /><ol><br /><li>encodeNewBuffer () uses new byte array to encode input string. </li><br /><li>encodeSynchronizedField () uses private field, guarded by synchronized{} block</li><br /><li>encodeThreadLocalField () uses ThreadLocal cache to encode input string</li><br /></ol><br /><br />Here is the code:<br /><pre class="csharpcode"><br /><span class="kwrd">long</span> encodeNewBuffer (String input) {<br /> final <span class="kwrd">byte</span> [] buffer = <span class="kwrd">new</span> <span class="kwrd">byte</span> [8]; <br /><br /> <span class="kwrd">return</span> f (buffer);<br />}<br /><br />/////////////<br /><br /><span class="kwrd">private</span> final <span class="kwrd">byte</span> [] buffer = <span class="kwrd">new</span> <span class="kwrd">byte</span> [8]; <br /><br />synchronized <span class="kwrd">long</span> encodeSynchronizedField (String input) {<br /> <span class="kwrd">return</span> f (buffer);<br />}<br /><br />/////////////<br /><br />ThreadLocal<<span class="kwrd">byte</span>[]> threadLocal = <span class="kwrd">new</span> ThreadLocal<<span class="kwrd">byte</span>[]>();<br />{<br /> threadLocal.set(<span class="kwrd">new</span> <span class="kwrd">byte</span> [8]);<br />}<br /><br /><span class="kwrd">long</span> encodeThreadLocalField (String input) {<br /> <span class="kwrd">byte</span> [] buffer = threadLocal.get();<br /> <span class="kwrd">return</span> f (buffer);<br />}<br /></pre><br /><br /><p>GC-based method is a winner:<br /><pre><br />encodeNewBuffer(): 4,108 sec. <br />encodeSynchronizedField(): 5,322 sec.<br />encodeThreadLocalField() : 5,411 sec<br /></pre>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-60160153779903877312009-11-09T09:38:00.001-08:002009-11-11T10:19:44.323-08:00Multi-threaded testing with JUnit. Switch to ConcJUnit .Our JUnit tests deal with components that use multiple threads. Unfortunately JUnit has no built-in support for this kind of testing. <br /><br />Lets consider a simple thread that interprets commands from input queue. In this example it can only understand "exit" command. All other inputs trigger an error.<br /><br /><pre><span class="s0"> <br /><a name="l8"> <br /></a><a name="l9"></a></span><span class="s1"><a name="l9">abstract class </a></span><span class="s0"><a name="l9">MainLoop </a></span><span class="s1"><a name="l9">extends </a></span><span class="s0"><a name="l9">Thread { <br /></a><a name="l10"> <br /></a><a name="l11"> </a></span><span class="s1"><a name="l11">private </a></span><span class="s0"><a name="l11">Queue<String> commands = </a></span><span class="s1"><a name="l11">new </a></span><span class="s0"><a name="l11">LinkedBlockingDeque<String>(); <br /><br /></a><a name="l12"> <br /></a><a name="l13"> </a></span><span class="s1"><a name="l13">public void </a></span><span class="s0"><a name="l13">run () { <br /></a><a name="l14"> </a></span><span class="s1"><a name="l14">while </a></span><span class="s0"><a name="l14">(</a></span><span class="s1"><a name="l14">true</a></span><span class="s0"><a name="l14">) { <br /></a><a name="l15"> String command = commands.poll(); <br /></a><a name="l16"> </a></span><span class="s1"><a name="l16">if </a></span><span class="s0"><a name="l16">(command.equals (</a></span><span class="s2"><a name="l16">"exit"</a></span><span class="s0"><a name="l16">)) <br /></a><a name="l17"> </a></span><span class="s1"><a name="l17">break</a></span><span class="s0"><a name="l17">; <br /><br /></a><a name="l18"> </a></span><span class="s1"><a name="l18">else</a></span><span class="s0"><a name="l18"> <br /></a><a name="l19"> reportError (</a></span><span class="s2"><a name="l19">"Unknown command: " </a></span><span class="s0"><a name="l19">+ command); <br /></a><a name="l20"> } <br /></a><a name="l21"> } <br /></a><a name="l22"> <br /></a><a name="l23"> </a></span><span class="s1"><a name="l23">public void </a></span><span class="s0"><a name="l23">add (String command) { <br /></a><a name="l24"> commands.add(command); <br /><br /></a><a name="l25"> } <br /></a><a name="l26"> <br /></a><a name="l27"> </a></span><span class="s1"><a name="l27">abstract void </a></span><span class="s0"><a name="l27">reportError (String error); <br /></a><a name="l28">} <br /></a><a name="l29"> <br /></a><a name="l30"></a></span><span class="s1"><a name="l30">public class </a></span><span class="s0"><a name="l30">MyTest { <br /></a><a name="l31"> <br /></a><a name="l32"> @Test <br /></a><a name="l33"> </a></span><span class="s1"><a name="l33">public void </a></span><span class="s0"><a name="l33">test() { <br /></a><a name="l34"> MainLoop thread = </a></span><span class="s1"><a name="l34">new </a></span><span class="s0"><a name="l34">MainLoop() { <br /><br /></a><a name="l35"> </a></span><span class="s1"><a name="l35">void </a></span><span class="s0"><a name="l35">reportError(String error) { <br /></a><a name="l36"> Assert.fail (error); <br /></a><a name="l37"> } <br /></a><a name="l38"> }; <br /></a><a name="l39"> thread.start(); <br /></a><a name="l40"> thread.add (</a></span><span class="s2"><a name="l40">"quit"</a></span><span class="s0"><a name="l40">); </a></span><span class="s3"><a name="l40">// sic! shoudl be "exit"</a></span><span class="s0"><a name="l40"> <br /><br /></a><a name="l41"> } <br /></a><a name="l42">} <br /></a><a name="l43"></a></span></pre><br /><br /><br />If we run the above test, JUnit completely ignores failure of "Commands Interpreter" thread (Although default uncaughtExceptionHandler prints exception to standard output):<br /><br /><pre><br /> [junit] Running test.MyTest<br /> [junit] Exception in thread "Commands Interpreter": java.lang.AssertionError: Unknown command: quit<br /> [junit] at org.junit.Assert.fail(Assert.java:91)<br /> [junit] at test.MyTest$1.reportError(MyTest.java:36)<br /> [junit] at test.MainLoop.run(MyTest.java:19)<br /> [junit] Tests run: 1, <b>Failures: 0, Errors: 0</b>, Time elapsed: 0.422 sec<br /></pre><br /><br />As you can see JUnit detected 0 errors. JUnit has at least two problems when it comes to multi-threaded tests:<br /><ol><br /><li>JUnit only cares about errors in its own thread. It completely ignores exceptions or assertion failures in child threads. </li><br /><li>Left over child threads may produce side effect on subsequent tests. In some cases failures in child threads may happen after the main thread is finished.</li><br /></ol><br /><br />We used various custom workarounds until it was clear that we need a generic solution for growing number of multi-threaded tests. Luckily we found an excellent tool - <a href='http://www.cs.rice.edu/~mgricken/research/concutest/concjunit/'>ConcJUnit</a> (AKA ConcuTest) written by Mathias Ricken.<br /><br />a) ConcJUnit is a drop-in replacement of JUnit. It includes standard JUnit classes and introduces own test runners. Just replace junit.jar by concutest.jar .<br /><br />b) ConcJUnit detects both failures in child threads and leftover threads.<br /><br />Switching to ConcJUnit produces expected result:<br /><br /><pre><br />java.lang.AssertionError: Unknown command: quit<br /> at org.junit.Assert.fail(Assert.java:91)<br /> at test.MyTest$1.reportError(MyTest.java:36)<br /> at test.MainLoop.run(MyTest.java:19)<br /> at test.MainLoop.run(MyTest.java:19)<br /></pre><br /><br />If you want to know more about ConcJUnit, read materials from <a href='http://www.cs.rice.edu/~mgricken/research/concutest/intro.shtml'>introduction</a> page. For example, the following <a href='http://www.cs.rice.edu/~mgricken/research/concutest/download/PPPJ2009-Ricken-ConcJUnit-2009-08-28.ppt'>presentation</a>.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-35284569485718414512009-11-05T08:38:00.000-08:002009-11-05T08:48:06.031-08:00GLOP"Consider a program — any program; let us call it GLOP. We begin designing our program by imagining that we have a machine with a hardware instruction that will perform GLOP (an idea that is not beyond the realm of possibility with today's microprogrammable computers) Thus, to write program GLOP, we merely write the instruction <pre>GLOP</pre> and we are done! Not only are we finished, but we are relatively certain that the resulting program is correct — assuming that the hardware primitive GLOP performs correctly (and if it doesn't, we can always blame it on the vendor)."<br /><br />Edward Yourdon "Techniques of program structure and design" 1975.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-78841481018082951422009-11-03T16:59:00.001-08:002009-11-03T16:59:55.390-08:002:5020/236.23 :-)Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-35813481102439288012009-10-08T21:27:00.001-07:002009-10-09T11:08:53.030-07:00sizeof and object padding in JavaConsider the following classes:<br /><br /><pre class="csharpcode"><br /><span class="kwrd">class</span> Struct1 {<br /> <span class="kwrd">byte</span> b1;<br />}<br /><br /><span class="kwrd">class</span> Struct2 extends Struct1 {<br /> <span class="kwrd">byte</span> b2;<br />}<br /><br /><span class="kwrd">class</span> Struct3 extends Struct2 {<br /> <span class="kwrd">byte</span> b3;<br />}<br /><br /><span class="kwrd">class</span> Struct4 extends Struct3 {<br /> <span class="kwrd">byte</span> b4;<br />}<br /><br /><span class="kwrd">class</span> Struct5 extends Struct4 {<br /> <span class="kwrd">byte</span> b5;<br />}<br /><br /><span class="kwrd">class</span> Struct6 extends Struct5 {<br /> <span class="kwrd">byte</span> b6;<br />}<br /><br /><span class="kwrd">class</span> Struct7 extends Struct6 {<br /> <span class="kwrd">byte</span> b7;<br />}<br /><br /><span class="kwrd">class</span> Struct8 extends Struct7 {<br /> <span class="kwrd">byte</span> b8;<br />}<br /><br /><br /><span class="kwrd">class</span> FlatStruct8 {<br /> <span class="kwrd">byte</span> b1, b2, b3, b4, b5, b6, b7, b8;<br />}<br /><br /></pre><br /><br />On my JVM (1.6.0_16 64-bit):<br /><br /><pre><br />sizeof (new Struct8()) = 80 (sic!)<br />sizeof (new FlatStruct8()) = 24<br /></pre><br /><br />Now, 24 makes sense (8 bytes of data + 2 x 8 bytes of object overhead). In case of Struct8, we can see that Java pads fields declared in each class to so that total block size will be multiple of 8.<br /><br />I used both <a href='http://sizeof.sourceforge.net/'>Instrumentation-based </a> and <a href='http://jroller.com/mipsJava/entry/sizeof_java_objects'>GC-based</a> sizeof () implementations to confirm this. <br /><br />On 32-bit JVM (1.6.0_04-b12) blocks are padded to multiple of 4.<br /><br /><pre><br />sizeof (new Struct8()) = 40 <br />sizeof (new FlatStruct8()) = 24<br /></pre><br /><br /><a href='http://www.javamex.com/tutorials/memory/object_memory_usage.shtml'>More</a> about padding.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-7330499106875917762009-07-27T15:16:00.000-07:002009-07-27T15:35:13.672-07:00Faster JUL loggingHere are the steps we followed to improve java.util.logging performance:<br /><ol><br /><br /><li>Switched from SimpleFormatter to our custom formatter. It boosted performance up to 6 times. Our formatter prints only time-of-day portion of timestamp (using custom formatting function). Our log files also omit class/method/thread context for each log message. It also uses custom MessageFormatter (based on StringBuilder).</li><br /><br /><li>Tweaked FileHandler to flush every 8K rather than after each message. This boosts performance up to 2.5 times.</li><br /><br /><li>We tried using Java7 logger package. We didn't notice considerable improvements for our usage patterns.</li><br /></ol><br /><br />Credit for much of this work goes to my colleague Nikolay Dul.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-48950784790429473722009-07-22T18:37:00.000-07:002009-07-27T15:16:06.732-07:00Java loggingAm I the only one who misses<br /><br /><pre><br />public void log (Level level, String msg, Object ... params)<br /></pre><br /><br />Instead of existing<br /><br /><pre><br />public void log (Level level, String msg, Object [] params)<br /></pre><br /><br />in java.util.logging.Logger ?<br /><br />P.S. Apparently not: enhancement request was filed back in <a href='http://bugs.sun.com/view_bug.do?bug_id=6540440'>2004</a>. I will see what I can do...Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-11257022727873986222009-06-13T18:16:00.000-07:002009-06-16T10:48:44.144-07:00Writing game bot in JavaCan Java be used to write a simple MMORPG <a href="http://en.wikipedia.org/wiki/Computer_game_bot">game bots</a>?<br /><br />Easily. Java has a class <a href="http://java.sun.com/j2se/1.5.0/docs/api/java/awt/Robot.html">java.awt.Robot</a> that is intended for writing automated tests and provides just what we need.<br /><br />First, lets create a bot that will keep an eye on your character's health status. The basic idea is very simple - when health status is running low, drink a healing potion. Repeat. This simple script covers your back and releases your mind from some routine actions. You will concentrate on attacking your opponent ;-)<br /><br /><pre>import java.awt.Color;<br />import java.awt.Robot;<br />import java.awt.Toolkit;<br />import static java.awt.event.KeyEvent.*;<br /><br />private Toolkit toolkit = Toolkit.getDefaultToolkit();<br />private Robot robot = new Robot();</pre><br /><br />We need two type of things:<br /><ol><br /><li>Inspect some area of the screen that shows your character health's status (usually appears as some sort of progress bar). We just need to know when some pixel inside the bar is getting black:<br /><pre>boolean isBlack (int x, int y) {<br /> Color c = robot.getPixelColor (x, y);<br /> return c.getRed() < 16 && c.getGreen() < 16 && c.getBlue() < 16;<br />}</pre></li><br /><li>Simulating keyboard action is also easy (we need to press a hotkey for your healing potions)<br /><pre>void key (int keyCode, long duration) throws InterruptedException {<br /> robot.keyPress(keyCode);<br /> Thread.sleep(duration);<br /> robot.keyRelease(keyCode);<br />}</pre></li></ol><br />Now the program itself is trivial:<br /><pre>while (true) {<br /> if (isBlack (90, 760)) <br /> key (VK_F1, 500);<br /> <br /> Thread.sleep (50); <br /> } <br /></pre><br /><br />Here I assume that your healing potions are assigned to hotkey F1 (KeyEvent.VK_F1) and coordinates {90, 760} show your health status and become black when health is below 25% (your can use Paint or some other program to analyze game's screen shot). <br /><br /><br />Similar approach can be used to automate other tedious MMORPG tasks (mining gold, fishing, or gaining experience). <br /><br /><br />P.S. Take a look at this guy's <a href="http://www.picamatic.com/view/1894033_1227048032450/">screen</a> (like a Boeing cockpit).<br /><br /><hr/><br />DISCLAIMER:Check your game's license before using this technique. It may be prohibited as something that gives unfair advantage over other players.<br /><hr/>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-30710518288997963002008-12-24T18:31:00.000-08:002009-02-28T13:24:13.063-08:00My favorite features in Scala<ul><br /><br /><li> Tupples in function results<br /><pre><br />def f(a: int, b: int) : (int, int) = (a/b, a%b)<br /></pre><br /><br /><li> type <tt>Nothing</tt> and <tt>Null</tt> (like NONE and ANY in Eiffel)<br /><li>Object definitions<br /><pre>object Singleton { }</pre><br /><li>Parameterless methods, that can be overridden by fields and vice versa. <br /><pre><br />class Circle(radius:double) {<br /> def diameter = 2*radius<br />}<br />...<br />new Circle(5).diameter<br /></pre><br /><br /><li>Subtyping of generics (so-called co-variant subtyping). I miss this feature in Java:<br /><pre><br />def Iterator[+E] ...<br />now Iterator[String] will assign to Iterator[AnyObj].<br /></pre><br /><br /><li>Pattern-matching expressions? Need to try it.<br /></ul><br /><br />Some of these I was missing since I programmed in Eiffel back in 1995.<br /><br />Features I am not so fond of:<br /><ul><br /><li>Case classes (a-la subtyped enums?)<br /></ul>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-58793031308632682612008-09-30T20:13:00.000-07:002010-07-08T12:28:33.951-07:00Mindstorms are backOnline game like this will be excellent for software job interview. <br /><div style="width: 148px; border: 1px solid #BBB; padding: 5px;background-color: white;"><a href="http://armorgames.com/play/2205/light-bot"><img alt="Cover voor Light Bot" src="http://www.gameroo.nl/covers/0000/0456/light-bot_medium.jpg" /></a></div>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-37098013442372800122008-09-12T07:29:00.000-07:002008-09-12T08:13:48.549-07:00Obfuscated codeSituation: Third-party client API library periodically fails with nasty exception:<br /><br /><pre><br />Caused by: cli.System.AccessViolationException: Attempted to read or write protected memory.<br />This is often an indication that other memory is corrupt.<br /> at cli.TalTrade.Toolkit.DataBlock.ReadData(Unknown Source)<br /> at cli.TalTrade.Toolkit.RawData.GetDataAsBlock(Unknown Source)<br /> ... 34 more<br /></pre><br /><br />After verifying that input parameters are legitimate I decided to take a look at the failing code. My trusty Reflector exposed the following:<br /><br /><!-- code formatted by http://manoli.net/csharpformat/ --><br /><pre class="csharpcode"><br /><span class="kwrd">private</span> <span class="kwrd">unsafe</span> <span class="kwrd">void</span> ReadData(<span class="kwrd">void</span> modopt(IsConstModifier)* pData)<br />{<br /> <span class="rem">// This item is obfuscated and can not be translated.</span><br /> ...<br /> <span class="kwrd">if</span> (*(((<span class="kwrd">int</span>*) (&gtal_handle + 4)))[4] < 6)<br /> <span class="kwrd">goto</span> Label_0038;<br /> <span class="kwrd">ushort</span> num5 = *((<span class="kwrd">ushort</span>*) (((*(((<span class="kwrd">int</span>*) (&gtal_handle + 4))) + *(((<span class="kwrd">int</span>*) (&gtal_handle + 4)))[6]) - 8) + 6));<br /> ... 100 more lines like these ...<br />}</pre><br /><br /><br />That code is supposed to read data block from memory queue and possibly format it as string. What kind of secret know-how can it be?<br /><br />Now instead of figuring out the problem by ourselves we are going to send this error to the vendor. What a waste of time on both sides.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-72570393300297587062008-08-14T06:54:00.000-07:002008-08-14T06:58:12.578-07:00Motorola RAZR repair<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHqyPwhyphenhyphen6tjvWcGjS8T26DNaiyHUMOBXJ-KRwXFfW_fcV4LLaVssfFT6D95aHYSgZuZIwnGiUDpKb_LLCAcAp9nfeebhFtHPzbObAvhTUtzrE-FKpMA5Xu_UbkkAaovNe6GlBetoMheMg/s1600-h/IMG_5743.JPG"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHqyPwhyphenhyphen6tjvWcGjS8T26DNaiyHUMOBXJ-KRwXFfW_fcV4LLaVssfFT6D95aHYSgZuZIwnGiUDpKb_LLCAcAp9nfeebhFtHPzbObAvhTUtzrE-FKpMA5Xu_UbkkAaovNe6GlBetoMheMg/s200/IMG_5743.JPG" border="0" alt=""id="BLOGGER_PHOTO_ID_5234371711365243058" /></a>Replaced keyboard in my cell phone. This is how it looked like at the worst moment.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-53855136208675222552008-08-05T20:04:00.000-07:002008-08-06T08:31:37.972-07:00Update on "direct vs. encapsulated field access times"Update to <a href="http://andy-malakov.blogspot.com/2008/08/direct-field-vs-method-access-times.html">earlier post</a> about performance of direct field versus method-encapsulated access. A friend of mine pointed to strange decrease in method access performance right before it is getting compiled to native code. So I went back and I enhanced output of the benchmark:<br /><pre class="csharpcode"><br /><br /> <span class="kwrd">static</span> <span class="kwrd">void</span> benchmark () throws Exception {<br /> FieldAccessor fv = <span class="kwrd">new</span> FieldAccessor ();<br /> MethodAssessor mv = <span class="kwrd">new</span> MethodAssessor ();<br /><br /> System.err.println (<span class="str">"\tTotal, Method, Field, Ratio"</span>);<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> n=1, total = 0; n < 10000000; n=2*n) {<br /> final <span class="kwrd">long</span> t0 = System.nanoTime();<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < n; i++) {<br /> <span class="kwrd">if</span> ((mv.getField() & 0x101) != 0)<br /> mv.setField(mv.getField()+1);<br /> <span class="kwrd">else</span><br /> mv.setField(mv.getField()+2);<br /> }<br /> final <span class="kwrd">long</span> t1 = System.nanoTime();<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < n; i++) {<br /> <span class="kwrd">if</span> ((fv.field & 0x101) != 0)<br /> fv.field = (fv.field+1);<br /> <span class="kwrd">else</span><br /> fv.field = (fv.field+2);<br /> }<br /> final <span class="kwrd">long</span> t2 = System.nanoTime();<br /><br /> <span class="rem">// dispay stats</span><br /> final <span class="kwrd">double</span> mvCycleTime = (<span class="kwrd">double</span>)(t1 - t0) / n;<br /> final <span class="kwrd">double</span> fvCycleTime = (<span class="kwrd">double</span>)(t2 - t1) / n;<br /> final <span class="kwrd">double</span> ratio = 100.0*fvCycleTime/mvCycleTime;<br /> total +=n;<br /> System.err.printf(<span class="str">"% 12d, %.3f, %.3f, %.2f%% %c\n"</span>,<br /> total, mvCycleTime, fvCycleTime, ratio,<br /> (fv.field == mv.getField() ? <span class="str">' '</span> : <span class="str">'!'</span>));<br /><br /> Thread.sleep(500);<br /> }<br /> }</pre><br />This code produces the following output (with -XX:+PrintCompilation):<br /><pre><small><br /> Total, Method, Field, Ratio<br /> 1, 5866.000, 1759.000, 29.99% <br /> 3, 5463.000, 405.500, 7.42% <br /> 7, 1925.500, 217.750, 11.31% <br /> 15, 1166.000, 156.250, 13.40% <br /> 31, 648.313, 123.938, 19.12% <br /> 63, 440.438, 110.250, 25.03% <br /> 127, 401.922, 103.547, 25.76% <br /> 255, 270.688, 106.156, 39.22% <br /> 511, 243.715, 95.785, 39.30% <br /> 1023, 225.525, 94.367, 41.84% <br /> 2047, 231.510, 142.017, 61.34% <br /> 4095, 232.009, 97.745, 42.13% <br /><span style="color:red;"> 2 tmp.TestFieldVsMethod$MethodAssessor::getField (5 bytes)</span><br /> 8191, 227.914, 97.593, 42.82% <br /><span style="color:red;"> 3 tmp.TestFieldVsMethod$MethodAssessor::setField (6 bytes)</span><br /> 16383, 238.234, 98.486, 41.34% <br /><span style="color:red;"> 1% tmp.TestFieldVsMethod::benchmark @ 44 (277 bytes)</span><br /> 32767, 13.628, 1.675, 12.29% <br /> 65535, 1.621, 1.666, 102.78% <br /> 131071, 1.611, 1.658, 102.88% <br /> 262143, 1.602, 1.655, 103.31% <br /> 524287, 1.599, 1.653, 103.41% <br /> 1048575, 1.897, 1.923, 101.40% <br /><span style="color:red;"> 4 java.lang.String::charAt (33 bytes)</span><br /> 2097151, 1.606, 1.652, 102.88% <br /> 4194303, 1.595, 1.652, 103.52% <br /> 8388607, 1.595, 1.652, 103.53% <br /> 16777215, 1.595, 1.655, 103.76% </small><br /></small></pre><br />Notice how ratio dives down at around 30,000 mark:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSakJ5QscbfBKR5XcFwq3FDOW6dj4GuhlzkDMrKkZqDGrXovmMJlD7xy3UYdPZM-WKLs5JxJ2KI0qS_UEWdrhGfqiNqG_Yl9ZV_l_4xzk_9OKog8WwV3qbAFx2thYMW8RWrMh3oUN3Y4U/s1600-h/fieldvsmethod.GIF"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSakJ5QscbfBKR5XcFwq3FDOW6dj4GuhlzkDMrKkZqDGrXovmMJlD7xy3UYdPZM-WKLs5JxJ2KI0qS_UEWdrhGfqiNqG_Yl9ZV_l_4xzk_9OKog8WwV3qbAFx2thYMW8RWrMh3oUN3Y4U/s400/fieldvsmethod.GIF" alt="" id="BLOGGER_PHOTO_ID_5231241290609597970" border="0" /></a><br /><br />What that probably means is that field access code is compiled faster:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgA80cm1ayaCifVrtW2OZ4kXWbnDN_GkBpCY9J7MmtVfZwicij2vR_kRWzDLkp_Sbv6l23SXB29wDg1ToJqoFyE5q76yuy1j8y5dyj2jB1LJ6y7U3POhpbSyC8jFYe46IsgC9frCsFBjyM/s1600-h/fieldvsmethod1.GIF"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgA80cm1ayaCifVrtW2OZ4kXWbnDN_GkBpCY9J7MmtVfZwicij2vR_kRWzDLkp_Sbv6l23SXB29wDg1ToJqoFyE5q76yuy1j8y5dyj2jB1LJ6y7U3POhpbSyC8jFYe46IsgC9frCsFBjyM/s400/fieldvsmethod1.GIF" alt="" id="BLOGGER_PHOTO_ID_5231241397249122354" border="0" /></a><br /><br />Without knowing much about hotspot inner workings, I expected to see a sharper decline. <br />I run this test on server version of Java 6 (1.6.0_10-beta-b25, 64 bit).</small></small>Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-10830676291918008702008-08-05T16:41:00.000-07:002008-09-02T11:41:14.517-07:00Java NIO with large data filesWe have a tool that sequentially reads financial data from massive multi-gigabyte files. It performs a lot of bytes-to-long, bytes-to-double, and utf-string conversions that were implemented in optimized fashion on top of <tt>BufferedInputStream</tt>. <br /><br />It turns out that replacing that code with NIO's <tt>MappedByteBuffer</tt> processes 2 Gb file <b>twice faster</b>, while memory consumption remains the same. <br /><br /><br /><strong>Update:</strong> I have been warned and it turned out to be true: Reading 20 Gb file shows reverse results. Now NIO version is <b>8x slower</b> than "old" stream-based approach. In this case I implemented paging because Java NIO cannot map file regions larger than 2Gb at once (Integer.MAX_VALUE).<br /><br />On Windows XP 8K is a sweet spot for BufferedInputStream buffer size. Allocating larger buffer actually reduces performance significantly. May be size of mapped region has similar optimum.Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-85899951408295976922008-08-05T10:27:00.000-07:002008-08-05T13:56:31.702-07:00Direct field vs. Method access timesIn my opinion there is no justification for using 'friendly' direct access to mutable class fields rather than getters and setters properties. I heard the following two arguments:<br /><br /><h3>1. It makes performance-critical code run faster.</h3><br /><br />False. There is no benefit of direct access in frequently executed code. Chart below compares access time of direct field access to getters and setters.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjF8KgLe01oWuFBVCyxSKzoAn48cgM1UwwyR8wv4Rhd5SPc5raDt1GQdnLR_ADOAwc-5F-bdoLZv4agUcehGv1YLl_-0aLeKzf9p8UO4FFvA-6GXgn1zPZYflZfBX0g0gGtlDdaHptbNSw/s1600-h/fieldvsmethod.GIF"><img style="cursor:pointer; cursor:hand;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjF8KgLe01oWuFBVCyxSKzoAn48cgM1UwwyR8wv4Rhd5SPc5raDt1GQdnLR_ADOAwc-5F-bdoLZv4agUcehGv1YLl_-0aLeKzf9p8UO4FFvA-6GXgn1zPZYflZfBX0g0gGtlDdaHptbNSw/s400/fieldvsmethod.GIF" border="0" alt=""id="BLOGGER_PHOTO_ID_5231100332848688290" /></a><br /><br />As you can see, after 20,000 calls there is no difference in performance. This threshold is controlled by <tt>-XX:CompileThreshold</tt> JVM argument. Core of the test:<br /><br /><pre class="csharpcode"><br /> final <span class="kwrd">long</span> t0 = System.nanoTime();<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < n; i++) {<br /> <span class="kwrd">if</span> ((mv.getField() & 0x101) != 0)<br /> mv.setField(mv.getField()+1);<br /> <span class="kwrd">else</span><br /> mv.setField(mv.getField()+2);<br /> }<br /> final <span class="kwrd">long</span> t1 = System.nanoTime();<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < n; i++) {<br /> <span class="kwrd">if</span> ((fv.field & 0x101) != 0)<br /> fv.field = (fv.field+1);<br /> <span class="kwrd">else</span><br /> fv.field = (fv.field+2);<br /> }<br /> final <span class="kwrd">long</span> t2 = System.nanoTime();<br /></pre><br /><br /><br /><h3>2. Direct field access reduces bureaucracy.</h3><br />Read: I am too lazy to write getters and setters. Well, Eclipse and Netbeans have <b>"Encapsulate Field"</b> and <b>"Generate getters and Setters"</b> commands. And BTW, what none of free IDEs have is <b>"Find where modified"</b> search for class fields.<br /><br />Having said that, I have no problems with <tt>public final</tt> fields :-)Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.comtag:blogger.com,1999:blog-8668853818783919860.post-14117202669148140362008-07-31T06:41:00.000-07:002008-08-04T18:50:23.915-07:00A bug in Sun's 64-bit hotspot compilerA colleague of mine found a scary bug in Sun's hotspot compiler (64 bit version). It is reproducible on several computers in our office (CPUs: 2x AMD Opteron, Intel E8400; OS: Windows XP Professional 64-bit). Here is the code:<br /><pre class="csharpcode"><br /><span class="kwrd">public</span> <span class="kwrd">class</span> FailingTest {<br /><br /> <span class="kwrd">private</span> final java.io.RandomAccessFile raf;<br /> <span class="kwrd">private</span> <span class="kwrd">long</span> sequence;<br /><br /> <span class="kwrd">public</span> FailingTest() throws Exception {<br /> java.io.File f = <span class="kwrd">new</span> java.io.File(<span class="str">"test.dat"</span>);<br /> f.delete();<br /> raf = <span class="kwrd">new</span> java.io.RandomAccessFile(f, <span class="str">"rw"</span>);<br /> f.deleteOnExit();<br /> }<br /><br /> <span class="kwrd">public</span> <span class="kwrd">void</span> test(<span class="kwrd">long</span> timestamp) throws Exception {<br /> <span class="rem">// This condition never happens but is required to</span><br /> <span class="rem">// reproduce this bug, so is unused timestamp argument</span><br /> <span class="kwrd">if</span> (timestamp < 0)<br /> System.exit(0);<br /><br /> final <span class="kwrd">long</span> valueToStore = sequence++;<br /> raf.seek(0);<br /> raf.writeLong(valueToStore);<br /> raf.seek(0);<br /> final <span class="kwrd">long</span> valueRead = raf.readLong();<br /> <span class="kwrd">if</span> (valueRead != valueToStore)<br /> System.err.println(<span class="str">"Error: Read value "</span> +<br /> valueRead + <span class="str">" != "</span> + valueToStore);<br /> }<br /><br /> <span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">void</span> main (String [] args) throws Exception {<br /> FailingTest t = <span class="kwrd">new</span> FailingTest();<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> ii = 0; ii < 1000000; ii++)<br /> t.test(ii);<br /> }<br />}<br /></pre><br />On my machine this test fails with the following output (I run it with <tt>-server -XX:CompileThreshold=100</tt>):<br /><pre style='font-color=red'><br />Error: Read value 4826 != 4825<br /></pre><br />The test fails every time with different numbers, with any version of 64-bit Sun JDK 1.6 we tried. This problem is NOT reproducible under 32-bit version and NOT reproducible on BEA's JRockit 1.6 (64 bit). Just verified that problem remains on the latest JVM available: build 1.6.0_10-beta-b25, Java HotSpot(TM) 64-Bit Server VM (build 11.0-b12, mixed mode).Anonymoushttp://www.blogger.com/profile/13571272251116627683noreply@blogger.com