| Author |
 Topic  |
|
jimmy.dias
BIAC Alum
USA
210 Posts |
 Posted - Jan 26 2005 : 2:16:45 PM
|
Hey all,
The Golgi pre-processing scripts are in a state of flux right now as I'm updating some of the scripts. Please hold off on running the following programs until further notice:
bputil
tralign
motioncorrect
coreg_normalize
normalize
coregister
smooth
Thanks,
Jimmy |
Edited by - jimmy.dias on Jan 27 2005 5:02:57 PM |
|
|
petty
BIAC Staff
USA
453 Posts |
Posted - Jan 27 2005 : 10:38:20 AM
|
| any word on how long this update is going to take? |
 |
|
|
mb
BIAC Alum
27 Posts |
Posted - Jan 27 2005 : 10:42:41 AM
|
Hi Jimmy,
While you're updating things, could you fix it so that the TR information isn't lost after motion correction? As it is, the TR info makes it to the header for the time slice corrected data but then goes missing, and bxh_eventstats won't run without it.
Thanks,
Mary Beth |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Jan 27 2005 : 10:55:05 AM
|
The following are ready to go:
tralign
motioncorrect
ETA for the rest are by the end of the day today at the latest.
MB,
I'm making changes in all of the scripts so any relevant (and correct) header information is retained after using the scripts.
Jimmy |
Edited by - jimmy.dias on Jan 27 2005 5:04:35 PM |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Jan 27 2005 : 4:57:05 PM
|
Two more are ready to go:
coreg_normalize
smooth
Keep in mind that bputil has not been updated to use the new scripts, so please call the scripts directly until I post otherwise.
Some of the changes that are being made (have been made):
- processing options can now be called from the commandline. (use --help for individual script usage information). If used, the parameters specified will override those options specified in your config file.
- output BXH files now retain all the information from the original input BXH files.
- input BXH files can be specified on the commandline instead of using runglobs or anatglobs. The disadvantage of using a runglob is that the script basically guesses which BXH to use if there are multiple bxh files in the directory. Please specify the location of each of the bxh files from now on. Sooner or later, the glob functionality will be removed as it causes many problems...
A simple coreg_normalize example:
coreg_normalize -a series001/series001.bxh run004_01/run004_01.bxh
- Data from a Siemens scanner can now be processed using the above scripts.
Jimmy |
Edited by - jimmy.dias on Jan 27 2005 5:08:31 PM |
|
|
|
mb
BIAC Alum
27 Posts |
Posted - Jan 28 2005 : 3:42:39 PM
|
Hi Jimmy,
Could you also keep a record of the time spacing in the dimensions field of the bxh file? bxh_eventstats needs that too ...
Thanks,
mb |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Jan 28 2005 : 4:20:09 PM
|
mb,
Thanks for the heads up. I've saved the old time spacing dimension values after coreg_normalize (instead of setting it to zero, woops!)
Jimmy |
|
|
|
petty
BIAC Staff
USA
453 Posts |
Posted - Jan 28 2005 : 4:24:25 PM
|
in the T dimensions the spacing should be the TR is ms:
<dimension type="t">
<size>210</size>
<origin>0</origin>
<gap>0</gap>
<spacing>1500</spacing> |
|
|
|
charles.michelich
BIAC Alum
USA
183 Posts |
Posted - Jan 28 2005 : 4:35:46 PM
|
The <units></units> element actual specifies the units of the spacing, origin, and gap (see BXH documentation). For the time dimension BIAC typically uses ms, but software written should read the actual units rather than assuming they are the expected units.<dimension type="t">
<units>ms</units>
<size>380</size>
<origin>0</origin>
<gap>0</gap>
<spacing>1500<spacing>
</dimension> |
|
|
|
mb
BIAC Alum
27 Posts |
Posted - Feb 03 2005 : 10:12:21 AM
|
Just curious, is it safe to use normalize?
mb |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Feb 04 2005 : 2:52:52 PM
|
hey all,
Just deployed:
bputil
normalize
A testable version of bputil is now available. The biggest change is that all data locations are stored in a centralized database. bputil queries the database when prompting you for selections such as functional run selection and anatomical run selection.
In order to update this database (and initially populate the database) after selecting an experiment, run bputil option 4.
Note that there are several processing options that are selectable via bputil option 6. The one most relevant is the option for backgrounding submitted jobs. By default, bputil will run without backgrounding (i.e. results will be printed to your screen). This option can/should be turned off when submitting large jobs. The force option uses dircopy --force to overwrite results when copying from Golgi back to the storage area. Disabling the force option basically uses the --master switch in dircopy (see dircopy --help for more details).
Please post questions, comments or concerns here or email me off list.
Thanks,
Jimmy
|
Edited by - jimmy.dias on Feb 04 2005 2:54:19 PM |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Feb 11 2005 : 4:06:46 PM
|
Hey all,
Apparently, I was a little too quick in deploying the smooth script on January 27th. As Evan was quick to point out, the smoothing function that was deployed had an old bug that was somehow reintroduced in the new scripts (the iterative counter variable being used instead of a fixed kernel width).
That bug has been once again fixed and the script has been re-deployed. Currently, users can either use bputil to submit batch jobs or call the underlying scripts (tralign,coreg_normalize, motioncorrect, smooth, normalize, and coregister) directly. Both methods have been tested.
As for an explanation of testing procedures and why this wasn't caught... Previously I only checked script outputs against a baseline only if major software revisions have been made. smooth and tralign had only minor cosmetic changes made to them, (argument handling in particular) and thus had not been put through our test framework. I realize that this error has caused almost two weeks of processing downtime and I apologize sincerely for it. In the future, all scripts with even minor changes will be sent through the test framework BEFORE being deployed. If anyone has any suggestions on how this process can be improved please let me know.
Much apologies,
Jimmy |
Edited by - jimmy.dias on Feb 11 2005 4:11:26 PM |
|
|
|
evan.gordon
BIAC Alum
12 Posts |
Posted - Feb 14 2005 : 09:20:58 AM
|
Hi Jimmy,
BPUtil's batching function appears to be malfunctioning. I tried to batch two subjects together and background the job, and bputil crashed immediately after adding the commands to the batch. The error given was:
Use of uninitialized value in concatenation (.) or string at /usr/local/bin/bputil line 961, <STDIN> line 13.
No new files or folders were generated by the script.
Evan |
|
|
|
evan.gordon
BIAC Alum
12 Posts |
Posted - Feb 14 2005 : 09:27:13 AM
|
Jimmy,
I'm also getting an error when I try to make BPUtil change experiments to Sequence.03. It gives this error:
Enter experiment name (-1 to cancel): Sequence.03
Use of uninitialized value in substr at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 297, <STDIN> line 2.
Use of uninitialized value in substitution (s///) at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 300, <STDIN> line 2.
Use of uninitialized value in pattern match (m//) at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 302, <STDIN> line 2.
Use of uninitialized value in concatenation (.) or string at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 306, <STDIN> line 2.
netpath is not in correct format at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 306, <STDIN> line 2.
Use of uninitialized value in concatenation (.) or string at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 282, <STDIN> line 2.
Use of uninitialized value in -d at /usr/opt/perl5/lib/5.8.0/BIAC_Common.pm line 283, <STDIN> line 2.
Use of uninitialized value in concatenation (.) or string at /usr/local/bin/bputil line 372, <STDIN> line 2.
ERROR! user gordon does not have net access to .
ERROR! check mount and network permissions
I do have the correct permissions to access Sequence.03. I don't get this error when trying to access any other experiment.
Evan |
|
|
|
jimmy.dias
BIAC Alum
USA
210 Posts |
Posted - Feb 14 2005 : 11:39:47 AM
|
Hi Evan,
quote:
BPUtil's batching function appears to be malfunctioning. I tried to batch two subjects together and background the job, and bputil crashed immediately after adding the commands to the batch. The error given was:
Use of uninitialized value in concatenation (.) or string at /usr/local/bin/bputil line 961, <STDIN> line 13.
The above is just a warning, which I'll clean up... Did it actually crash or did it just return to the main menu (the correct behavior)? The batch jobs you've submitted are actually running. You can check this by a
ps -fu <username> at the prompt.
The shell script (running per job submitted) is called instruction_file.sh and is located in a temporary directory on Golgi (in /data/users/<username>/bputil). To look for bputil jobs in particular, the following would work:
ps -ef | grep instruction_file.sh
You can also look at the shell script by using less <filename> or more <filename> at the prompt to see what commands the script contains.
Jimmy |
Edited by - jimmy.dias on Feb 14 2005 3:38:37 PM |
|
|
|
evan.gordon
BIAC Alum
12 Posts |
Posted - Feb 14 2005 : 12:00:26 PM
|
I also thought it was just a warning, but three hours have now gone by without any files or folders being written out.
Evan |
|
|
|
Topic |
|
Golgi pre-processing scripts
