https://wiki.jlab.org/cugwiki/index.php?title=Computing_and_file_services&feed=atom&action=historyComputing and file services - Revision history2024-03-28T21:34:08ZRevision history for this page on the wikiMediaWiki 1.35.7https://wiki.jlab.org/cugwiki/index.php?title=Computing_and_file_services&diff=614&oldid=prevKuhn at 15:04, 15 November 20122012-11-15T15:04:56Z<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 15:04, 15 November 2012</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l79" >Line 79:</td>
<td colspan="2" class="diff-lineno">Line 79:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>question or suggestion for new features, please feel free to send email </div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>question or suggestion for new features, please feel free to send email </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>to scicomp@jlab.org.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>to scicomp@jlab.org.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">'''NOTE:''' We recently completed a survey of scientific computing at Jefferson Lab. The results of the survey can be found here: [[Scientific Computing Users Survey]]</ins></div></td></tr>
</table>Kuhnhttps://wiki.jlab.org/cugwiki/index.php?title=Computing_and_file_services&diff=597&oldid=prevKuhn: Created page with 'Jefferson Lab Scientific Computing Group will introduce a significant upgrade to the existing mass storage and batch processing systems in early November this year. Beginning i…'2012-09-28T15:31:33Z<p>Created page with 'Jefferson Lab Scientific Computing Group will introduce a significant upgrade to the existing mass storage and batch processing systems in early November this year. Beginning i…'</p>
<p><b>New page</b></p><div>Jefferson Lab Scientific Computing Group will introduce a significant <br />
upgrade to the existing mass storage and batch processing systems in <br />
early November this year. Beginning in early October, users will notice <br />
warnings and informational messages about the pending upgrade when they <br />
execute existing commands. The Scientific Computing group has invested a <br />
lot of development effort to keep syntax and behavior of user commands <br />
in the upgraded system similar to those of existing commands even though <br />
underlying systems have been changed greatly. The following list <br />
describes the noteworthy changes in the upgraded system:<br />
<br />
1. User certificate system.<br />
<br />
The current authentication system uses self-signed certificates with <br />
exceptionally long expiration dates. In accordance with <br />
industry-standard practices, and to improve the security of our system, <br />
new user certificates must be signed by the Jefferson Lab central <br />
signing authority, and will be valid for two years.<br />
<br />
All users will be required to replace their certificates by running a <br />
command line utility on any centrally managed Linux machine such as <br />
jlabl1. The general availability of this tool will be sent out <br />
initially through email and web postings, followed later by warning <br />
messages from existing commands.<br />
<br />
2. New disk cache and volatile management systems.<br />
<br />
The new cache and volatile systems are based on a Lustre global file <br />
system, which provides better performance and better scalability. The <br />
volatile system is already in production serving as a global scratch <br />
disk area with enabled quota and reservation capabilities. The cache <br />
system will serve as a front end to the tape library. The new cache <br />
system offers users the ability to explicitly pin cached files. <br />
Moreover, each file in a DST storage group will be assigned a relatively <br />
long pin time. Users can modify the pins, obviating the current <br />
practice of manually managing DST cache files.<br />
<br />
3. Upgraded Jasmine system.<br />
<br />
The upgraded back-end system is actually already in production. Some <br />
notable improvements include: use of MD5 checksums to better ensure data <br />
integrity, opportunistic direct transfer between network file-systems <br />
and the tape library to improve throughput and reduce network traffic, <br />
distributed load-balancing and prioritization to provide fair access to <br />
the tape library, automatic background duplication of all raw data to <br />
avoid catastrophic data loss, automated error recovery to improve tape <br />
library availability, and improved auditing of the systems to provide <br />
better reporting capabilities.<br />
<br />
The user tools have been re-written to deliver more useful information <br />
during operation as well as on system error. Please note that the <br />
output of jput and jget will change significantly. Consult the <br />
Scientific Computing wiki if you have scripts that depend upon the <br />
output of these commands.<br />
<br />
4. Upgraded Auger system.<br />
<br />
The Auger system has been re-engineered to deliver better utilization of <br />
the farm nodes, cache disks and the tape library system. All Auger <br />
commands retain the current syntax. However, the output of a new Auger <br />
command will be in XML format.<br />
<br />
5. Scientific Computing Web Portal.<br />
<br />
A new web portal will be deployed in the same time frame. It will <br />
utilize modern web technologies such as AJAX and JQuery. It will offer <br />
the user a better web experience, and will feature more data mining <br />
capabilities.<br />
<br />
This upgrade is aiming to provide users a better experience using <br />
Jefferson Lab scientific computing facilities in addition to deliver the <br />
best utilization possible of the facilities. The underlying software has <br />
been either re-implemented or completely redesigned using current <br />
best-practice software techniques to improve reliability, performance <br />
and maintainability of our whole software system.<br />
<br />
More information about this upgrade can be found in the wiki page <br />
https://wiki.jlab.org/cc/external/wiki/index.php/Scientific_Computing <br />
under the entries of "Upcoming Software Upgrade". If you have any <br />
question or suggestion for new features, please feel free to send email <br />
to scicomp@jlab.org.</div>Kuhn