<p>In Pipeline Pilot, specify a sequence file. Supported file formats include BSML, PIR, FASTA, Genbank, and SwissProt. In Discovery Studio, choose from the list of sequences in all open Sequence Windows.</p>
<!-- <DSClientParameterType=AasequenceType> -->
data\Fasta\tyrosineKinase.fasta
<p>
Name of a NCBI database for searching:</p>
<ul>
<li><b>nr</b>: non-redundant protein sequences.</li>
<li><b>refseq_protein</b>: reference proteins.</li>
<li><b>swissprot</b>: Swissprot protein sequences.</li>
<li><b>pat</b>: patented protein sequences.</li>
<li><b>pdb</b>: Protein Data Bank proteins.</li>
<li><b>env_nr</b>: environmental samples.</li>
</ul>
<p>
<b>See Also:</b></p>
<ul>
<li>Check the NCBI BLAST web site for more database information.</li>
</ul>
User Specified Database
nr
refseq_protein
swissprot
pat
pdb
env_nr
User
Allows the specification of a database other than those known of at the time of the protocol release. Requires that Custom is chosen as the Input Database. It is the user's responsibility to specify a valid database name.
SetInvalidDetails('Parameter must not be blank when Input Database is set to User');
trim(Parameter()) ne ""
SetDisabledDetails('Enabled when Input Database is set to User');
parameter('Input Database') rlike 'user'
<p>An organism name to limit search.</p>
<p><i>Organism</i> names should preferably be the scientific name of the organism. However, a number of common synonyms are supported, as specified in the NCBI Taxonomy database.</p>
<p>Examples: "human", "Homo sapiens", "house mouse", "Mus musculus" or "Caenorhabditis elegans"</p>
<p>Threshold expectation value for hits to be reported.</p>
<p>This is the E-value from the Karlin-Altschul equation. The default value (10) means that 10 such matches are expected to be found merely by chance. If the statistical significance ascribed to a match is greater than this threshold, the match will not be reported. A lower <i>E-value Cutoff</i> will typically result in fewer, but more significant hits. </p>
<p><b>See Also:</b></p>
<ul><li>Karlin, S. and Altschul, S.F. Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes. <i>Proc. Natl. Acad. Sci. USA</i> (1990), 87:2264-2268.</li></ul>
SetInvalidDetails("Must be a number greater than 0");
Parameter() >0;
10
Number of sequences to be reported.
SetInvalidDetails("Must be a number greater than 0");
Parameter() >0;
100
Contains less commonly used parameters.
Word Size
URL
Scoring Matrix
Gap Penalties
Gapped Alignment
Use Composition Based Statistics
Filter Low Complexity
<p>Scoring matrix used in searching.</p><p> The matrix assigns a score for aligning any possible pair of residues:</p>
<ul>
<li><b>BLOSUM45</b>: for particularly long and weak similarities.</li>
<li><b>BLOSUM62</b>: for detecting most weak protein similarities. </li>
<li><b>BLOSUM80</b>: for sequence length between 50 and 85 residues.</li>
<li><b>PAM70</b>: for sequence length between 35 and 55 residues.</li>
<li><b>PAM30</b>: for sequence length shorter than 35 residues.</li>
</ul>
BLOSUM45
BLOSUM62
BLOSUM80
PAM70
PAM30
Value pair for the gap open and gap extension penalties. Increase the <i>Existence</i> to prevent a gap from being opened. Increase the <i>Extension</i> to prevent a gap from being extended. The recommended value for each <i>Scoring Matrix</i> is:
<ul>
<li><b>Existence: 15 Extension: 2</b> for BLOSUM45</li>
<li><b>Existence: 11 Extension: 1</b> for BLOSUM62</li>
<li><b>Existence: 10 Extension: 1</b> for BLOSUM80</li>
<li><b>Existence: 10 Extension: 1</b> for PAM70</li>
<li><b>Existence: 9 Extension: 1</b> for PAM30</li>
</ul>
if parameter("Scoring Matrix") eq 'BLOSUM62' then
append(legalvals,'Existence: 9 Extension: 2');
append(legalvals,'Existence: 8 Extension: 2');
append(legalvals,'Existence: 7 Extension: 2');
append(legalvals,'Existence: 12 Extension: 1');
append(legalvals,'Existence: 11 Extension: 1');
append(legalvals,'Existence: 10 Extension: 1');
elsif parameter("Scoring Matrix") eq 'PAM30' then
append(legalvals,'Existence: 7 Extension: 2');
append(legalvals,'Existence: 6 Extension: 2');
append(legalvals,'Existence: 5 Extension: 2');
append(legalvals,'Existence: 10 Extension: 1');
append(legalvals,'Existence: 9 Extension: 1');
append(legalvals,'Existence: 8 Extension: 1');
elsif parameter("Scoring Matrix") eq 'PAM70' then
append(legalvals,'Existence: 8 Extension: 2');
append(legalvals,'Existence: 7 Extension: 2');
append(legalvals,'Existence: 6 Extension: 2');
append(legalvals,'Existence: 11 Extension: 1');
append(legalvals,'Existence: 10 Extension: 1');
append(legalvals,'Existence: 9 Extension: 1');
elsif parameter("Scoring Matrix") eq 'BLOSUM80' then
append(legalvals,'Existence: 8 Extension: 2');
append(legalvals,'Existence: 7 Extension: 2');
append(legalvals,'Existence: 6 Extension: 2');
append(legalvals,'Existence: 11 Extension: 1');
append(legalvals,'Existence: 10 Extension: 1');
append(legalvals,'Existence: 9 Extension: 1');
elsif parameter("Scoring Matrix") eq 'BLOSUM45' then
append(legalvals,'Existence: 13 Extension: 3');
append(legalvals,'Existence: 12 Extension: 3');
append(legalvals,'Existence: 11 Extension: 3');
append(legalvals,'Existence: 10 Extension: 3');
append(legalvals,'Existence: 15 Extension: 2');
append(legalvals,'Existence: 14 Extension: 2');
append(legalvals,'Existence: 13 Extension: 2');
append(legalvals,'Existence: 12 Extension: 2');
append(legalvals,'Existence: 19 Extension: 1');
append(legalvals,'Existence: 18 Extension: 1');
append(legalvals,'Existence: 17 Extension: 1');
append(legalvals,'Existence: 16 Extension: 1');
end if;
SetLegalValues(legalvals);
Existence: 11 Extension: 1
<p>Specifies whether to use gapped BLAST or ungapped BLAST algorithm.</p>
<p>The ungapped BLAST algorithm invokes the older, ungapped style of alignment, according to the Karlin-Altschul statistics.</p>
<p><b>See Also:</b></p>
<ul><li>Karlin, S. and Altschul, S.F. Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes. <i>Proc. Natl. Acad. Sci. USA</i> (1990), 87:2264-2268.</li></ul>
True
False
Use the amino acid composition of the individual database sequences involved in reported alignments when calculating the E-values. This improves the accuracy of E-values and thus reduces the number of false positives.
SetInvalidDetails('Composition Based Statistics requires Gapped Alignment being set to TRUE');
Parameter('Gapped Alignment') eq 'True';
True
False
Specifies whether to mask off segments of the query sequence that have low compositional complexity. Filtering can remove hits that are statistically significant but biologically uninteresting (e.g., hits against common acidic-, basic- or proline-rich regions).
True
False
Specifies the length (either 2 or 3) of the initial matching segment. A larger value tends to yield more accurate results, but takes more computation time.
2
3
URL for online BLAST server
https://blast.ncbi.nlm.nih.gov/Blast.cgi
Map parameters for Discovery Studio backwards compatibility.
Database,Input Database;Matrix,Scoring Matrix;Gap Cost,Gap Penalties;Number of Sequences in Output,Maximum Hits;Expectation Value,E-value Cutoff;
Lists the set of temporary files which will be converted by the network to true temporary file names
This group contains parameters specific to protocols containing Reporting components.
Reporting Stylesheet
Specifies the name of a Stylesheet to be used by this protocol if any of the Reporting components have a Style selected in the <i>Format</i> and <i>Position</i> parameters.
{42691EC8-0CE0-4DB9-8B3C-5379CBB967A7}
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Add Protocol Comment Here
Add Protocol Comment Here
ProtocolNode
Component Revision
100
0
70 -50
By Ref
0
100
Searches protein databases at the NCBI with a query sequence using BLAST
<p>Uses the BLAST server at the National Center for Biotechnology Information (NCBI). BLAST is a statistically driven search method that finds regions of similarity between a query sequence and database sequences and produces gapped or ungapped alignments of these regions.</p>
<p>The NCBI maintains up-to-date protein databases that enable you to perform sequence similarity searches over the internet against all publicly available sequence data.</p>
None
None
None
The server version when this component was registered
20.1.0.2200
Protocol Sticky Notes
Protocol Options
Make Shortcut by Default
Owner Access Only
Protocol Scale
100
The component this component is based upon.
EmptyProtocol
Component Comment
A top-level component for building protocols.
Component Usage
The user should customize this description and usage to describe
the functioning of their derived protocol.
BLAST Search (NCBI Server)
Protocol AutoLayout
1
The user who registered this component.
HKH1
The date this component was registered.
Mon Apr 20 16:53:17 2020
{B11F2E16-0879-4FF8-8C4D-0796C11617B7}
A comma separated list of global properties which will be automatically
exported if this protocol is published as a web service.
Protocol ShowComment
true
Local Component Identifier
-1
OriginalPackage
scitegic/dscore
The original path within the database of this component.
Protocols\BIOVIA Discovery Studio\Discovery Studio\Macromolecules\Sequence Analysis\BLAST Search (NCBI Server)
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
use strict;
use Proteomics::BLASTCommon;
sub onInitialize {
my $context = shift;
my $gpRef = $context->getGlobalProperties->getHashRef;
#
my $pn = $gpRef->{'Gap Penalties'};
if($pn=~/(\d+).+(\d+)/) {
$gpRef->{'GapOpeningPenalty'} = $1;
$gpRef->{'GapExtensionPenalty'} = $2;
} else {
die("Unknown Gap Penalties $pn");
}
if (uc($gpRef->{'Gapped Alignment'}) eq 'TRUE') {
$gpRef->{'Use Composition Based Statistics Flag'} = $gpRef->{'Use Composition Based Statistics'}
} else {
$gpRef->{'Use Composition Based Statistics Flag'} = 'False';
}
my $db = $gpRef->{'Input Database'};
if(lc($db) eq "user"){
$gpRef->{'Database'} = $gpRef->{'User Specified Database'};
}else{
$gpRef->{'Database'} = $db;
}
return pilot::READYFORINPUTDATA;
}
sub onProcess {
my $context = shift;
my $data = shift;
return pilot::READYFORINPUTDATA;
}
sub onFinalize {
my $context = shift;
}
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/dscore
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
If True, the Perl interpreter instance for the component is terminated in the onFinalize event.
True
False
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Nov 06 19:45:37 2006
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Comment
Loads a class to provide component behavior.
Component Usage
Loads a class to provide component behavior.
Set the ports and icon as appropriate.
Set Parameters
90 -95
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_comp_dev.pdf')">Perl Component Development Guide</a></li>
</ul>
None
None
None
The server version when this component was registered
6.0.1.4
Local Component Identifier
0
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
WarningMsg is defined and WarningMsg ne "";
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
1890 -455
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
42
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
<p>Short text send to the DS client.</p>
<strong>Note:</strong>
<ul>
<li>Text is limited to 32 characters.</li>
</ul>
Client Message Final
<p>Text send to the DS Client.</p>
<strong>Note:</strong>
<ul>
<li>Text length is not limited.</li>
</ul>
Parameters controlling the DS report.
Report Error
Remove Duplicates from Summary
Report Detail
Report Warning
Report Summary
Text added to <strong>Summary</strong> section in the DS report.
Text added to <strong>Detail</strong> section in the DS report.
Al1HAAFBVGV4dCBhZGRlZCB0byA8c3Ryb25nPldhcm5pbmc8L3N0cm9uZz4gc2VjdGlvbiBpbiB0aGUgRFMgcmVwb3J0LixTdHJpbmdUeXBlAAgAAAABAQABAAQA/AAAAAAAAAAAADxSZXBvcnQgV2FybmluZwPMU8+7BAA=
StringType
StringType
Text added to <strong>Warning</strong> section in the DS report.
WarningMsg
Text added to <strong>Error</strong> section in the DS report.
<strong>Note:</strong>
<ul>
<li>A specified text will change the run status to failure.</li>
</ul>
When set to True, all duplicate <strong>Summary</strong> lines in the DS report are removed.
True
False
Contains less commonly used parameters.
Debug Level
Debug Message
Message to send to the log file (if log is specified)
User can set the debug level for the debug message.
1
2
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The date this component was registered.
Mon Feb 26 16:26:52 2018
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Messaging
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{8F288684-93BC-4712-BC9A-73930339BDEB}
2190 -455
By Ref
1
100
Sends text to the client and/or to the DS report
<p>Messages can be sent to the client or text can be specified for the DS report.</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
18.1.0.1604
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
Local Component Identifier
43
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\DS Messaging
<p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.</p><p>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
#rtcForEachCategory := (Parameter('RunToCompletion') eq 'True - For Each Category');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
elsif (#rtcForEachCategory) then
SetDisabledDetails("This parameter is not applicable when 'RunToCompletion' is set to 'True - For Each Category'");
end if;
#parallel and not #rtcForEachCategory;
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsFail
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Specifies the username to pass the proxy server authentication.
Proxy Authentication Username
Specifies the password to pass the proxy server authentication.
Proxy Authentication Password
Fetch Hit Sequences
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
1590 -635
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them. For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut, so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating Shortcuts</a>.
</p>
Generic
None
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
7.9.0.30
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Tue Jan 26 10:02:05 2010
Local Component Identifier
29
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
When a General or DataType error occurs during processing:
LeaveData: The data is passed unalterd
ClearData: The data is cleared before being passed (an empty data item)
DeleteData: The data is deleted and not passed
LeaveData
ClearData
DeleteData
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
CustomErrorText
the action to perform
PassDataToPort
the port to pass data to
1
COMPONENTNODE
The user who registered this component.
scitegic/generic
The date this component was registered.
Thu Sep 28 16:34:08 2006
No-op
The component this component is based upon.
Evaluate Expression
0
{6E468A38-A6F7-4371-B65B-501A3ED4C0D9}
465 -95
100
Sends all incoming records out the Pass port unchanged
<p>Useful in subprotocols to capture the point of input. For example, if you wish to perform two different sequences of operations on the same input data, use No-op as the first component of the subprotocol and connect both pipes to the No-op output port.</p>
<p>You can also use this component to simplify the layout for complex protocols. For example, if you have three Reader components that need to be connected to three Viewer components, placing a No-op in the middle reduces the number of required connections and clarifies the logic.</p>
Generic
Generic
None
The server version when this component was registered
6.0.1.1
Local Component Identifier
6
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\No-op
<p>The name of the property to hold the data record count. If the name begins with "@", this is a global property name and is updated each time a data record is received. Otherwise, it is a local property name and a new property by that name will be created on each data record and given the current value (i.e., the count).</p>
Start Index
Check if Property Already in Use
@NumSeqs
The index (i.e., number) used to start the sequence.
1
Generates an error if the property already exists. This helps prevent overwriting data created by another component. Default is "True".
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
The user who registered this component.
scitegic/generic
The date this component was registered.
Tue Sep 13 07:16:44 2016
{7BF3D951-BF75-70A2-8B2C-AD8909791909}
Count and Index Data
COMPONENTNODE
The component this component is based upon.
Count and Index Data
0
1365 -95
100
Counts the number of data records and puts the count in a property
<p>Use this component to create a numerical index of data records and to keep track of the total number of records. The value of <i>Counter Property</i> is incremented each time a data record enters the component.</p>
<p>This component can be used in two modes:</p>
<ul>
<li>If <i>Counter Property</i> is the name of a local property (does not start with "@"), a new property by that name is created on each data record and given the value of the current count.</li>
<li>If <i>Counter Property</i> is the name of a global property (starts with "@"), then each data record that passes through causes this global record count to be incremented, but values are not added to the passing records.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
17.1.0.119
Local Component Identifier
16
The original path within the database of this component.
Components\Data Access and Manipulation\Data Manipulators\Count and Index Data
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
/*make sure display ID is the same as the accessionNumber in the hit file-
this may not be the case for sequences which had no gi originally or where the database name needed removing for EFetch*/
if(accessionNumberFull is defined)then
debugMessage("full accession ". accessionNumberFull . " accessionNumber now is " . accessionNumber . " and display ID is " . displayID);
accessionNumber := accessionNumberFull;
else
debugMessage("full accession was not retained, accessionNumber is " . accessionNumber . " and display ID is " . displayID);
end if;
displayID := accessionNumber;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1665 -95
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
14
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The filename for the destination sequences
Maximum
Use Original FASTA ID
If File Exists
Character Encoding
NewlineStyle
Sequence Line Length
$(RunDirectory)/blastseqs.fasta
The maximum number of sequences to write (all if value is empty)
Specifies the maximum number of characters per line for the sequence output.
If you specify 0, the whole sequence will be printed on a single line.
60
What to do if the given destination file already exists
Overwrite
Append
Halt
If the sequence data record has an "originalFastaID" property, use its value as the ID in the FASTA record.
True
False
<p>Indicates the character set that will be used within the file.</p>
<ul>
<li><b>ANSI</b> is the default and specifies a character set where each character is defined by a single byte. This allows the Latin-1 (ISO-8859-1) character set to be written to the file. Virtually all applications can read these files.</li>
<li><b>UTF-8</b> specifies a variable length character set. Each character is represented by 1 to 6 bytes. This allows writing out all extended characters specified in the Unicode character sets while maintaining reasonably lengthed files. Many applications do not support UTF-8 files.</li>
<li><b>Unicode</b> and <b>Unicode big endian</b> are sets where each character is represented by exactly 2 bytes (also known as UCS-2). Depending on which is used will determine the order in which the bytes appear for each character. Many applications do not support Unicode or Unicode big endian files.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information about each of these encoding styles, go to <a href="javascript:DoLink('http://www.unicode.org/')">http://www.unicode.org/</a></li>
</ul>
UseByteOrderMark
ANSI
UTF-8
Unicode
Unicode big endian
<p>The byte order mark (BOM) is an optional Unicode character written to the start of a file or data stream that can be used to indicate the character encoding of the following data. Spefically it can be used to indicate the various UTF-8 and UTF-16 encodings.
</p>
<p>
BOMs are more widely used in a Windows setting. (For example, Windows Notepad includes a BOM in UTF-8 files.)
</p>
<p>
Setting options are:
</p>
<dl>
<dt>Always</dt>
<dd>For non-ANSI encodings, the BOM is always written at the start of the output stream.</dd>
<dt>Never</dt>
<dd>No BOM is written at the start of the output stream.</dd>
<dt>Native</dt>
<dd>This is equivalent to "Always" on Windows and "Never" on Linux.</dd>
</dl>
SetDisabledDetails(
'To enable '.ParameterName().', set Character Encoding to a non-ANSI value');
Parameter('Character Encoding') ne 'ANSI';
Always
Never
Native
<p>
Set this value to define the line ending character(s) in the output file. The default value is the Windows-standard carriage return plus line feed (CR+LF), but you may need to change this, especially in a non-Windows environment. Options are:</p>
<ul>
<li>CR+LF (Traditional Windows text file line ending)</li>
<li>LF (Line feed, '\n', 0x0A)</li>
<li>CR (Carriage return, '\r', 0x0D)</li>
<li>Native (CR+LF on Windows, LF on Linux)</li>
</ul>
CR+LF
LF
CR
Native
Aliased by the <b>If File Exists</b> parameter on the Parameters tab.
Do not modify this value.
$(If File Exists)
Aliased by <b>Character Encoding</b> parameter on the Parameters tab.
Do not modify this value.
$(Character Encoding)
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Thu Mar 10 12:46:10 2011
{EC23DB98-971A-49EF-8E9F-5A23D33BE8FB}
FASTA Sequence Writer
1965 -95
DOCUMENTNODE
The component this component is based upon.
Plug In
0
100
Writes a FASTA sequence file.
Creates a flat file entry for each sequence data record. There is a parameter to limit the number of sequences written. There is no fail port.
<strong>See Also:</strong> <a href="javascript:DoLink('sciapps/scitegic/seqanal/docs/seqanal.pdf')">Sequence Analysis User Guide</a>
Generic
None
None
The server version when this component was registered
8.4.5.21
Local Component Identifier
3
The original path within the database of this component.
Components\Biology\Sequence Analysis\Writers\FASTA Sequence Writer
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
/* gi|3402063|pdb|1A6V|L */
#status := false;
AccessionNumberFull := accessionNumber;
debugMessage("accession number before EFetch is " . AccessionNumber);
#hitid := RSPLIT(accessionNumber,'\|','g');
/*keep this */
/* does the accession contain a gi number? */
if(#hitid[1] eq "gi")then
primaryID := #hitid[2];
#status := True;
debugMessage("got primary ID " . primaryID . " from accession " . accessionNumber);
else
debugMessage("no gi in accession " . accessionNumber);
/*so need to check if there are 2 fields which we assume is dbname|accessionNumber ... actually 3 is returned for db|accession|
The EFetch apparently needs plain accessionNumber */
if(NumValues(#hitid) = (2 or 3))then
if(NumValues(#hitid) = 3 and #hitid[1] rlike "pdb")then
accessionNumber := #hitid[2] . "_" . #hitid[3];
#status := true;
elsif(NumValues(#hitid) = 3 and #hitid[2] eq "") then /*weird thing like the prf||2007441A I found */
accessionNumber := #hitid[3];
debugMessage("WEIRD accession");
#status := true;
else
accessionNumber := #hitid[2];
#status := true;
end if;
elsif(NumValues(#hitid) = 1)then
/*I think we have to assume this is the plain accession number and continue*/
#status := true;
else
debugMessage("number of values in hit ID is " . NumValues(#hitid));
end if;
end if;
if(#status)then
debugMessage("accessionNumber now is " . accessionNumber);
/* sanity check*/
if( accessionNumber eq "")then
#status := false;
end if;
end if;
#status;
An expression evaluated only when the component
is first inialized (when the first data record enters).
(Since there is no data available at initialization
time, only expressions referencing global
properties are allowed.)
An expression evaluated only when the
component is finalized (once the last data
record has passed).
*Since there is no data available at finalization
time, only expressions referencing global
properties are allowed.
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
Comment
Allows you to filter using the result from a PilotScript expression. The expression you specify is evaluated for each incoming data record, and is evaluated to a True or False value. If multiple statements are present, then the value of the last statement is used.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:18 2006
Component Usage
The 'Expression' is evaluated for each incoming data record. The last statement in the expression evaluates to a boolean value (true/false or nonzero/zero) and determines the fate of the data record: Records which evaluate to 'true' are passed out the Pass port; those which evaluate to 'false' are passed out the 'Fail' port.
Input - Data records
Output Pass - Data records which evaluate 'true'
Output Fail - Data records which evaluate 'false'
set primaryID
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -275
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
None
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
0
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
6.0.2.0
<p>The name of the property to hold the data record count. If the name begins with "@", this is a global property name and is updated each time a data record is received. Otherwise, it is a local property name and a new property by that name will be created on each data record and given the current value (i.e., the count).</p>
Start Index
Check if Property Already in Use
@NumIDs
The index (i.e., number) used to start the sequence.
1
Generates an error if the property already exists. This helps prevent overwriting data created by another component. Default is "True".
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
The user who registered this component.
scitegic/generic
The date this component was registered.
Tue Sep 13 07:16:44 2016
{7BF3D951-BF75-70A2-8B2C-AD8909791909}
Count and Index Data
COMPONENTNODE
The component this component is based upon.
Count and Index Data
0
765 -275
100
Counts the number of data records and puts the count in a property
<p>Use this component to create a numerical index of data records and to keep track of the total number of records. The value of <i>Counter Property</i> is incremented each time a data record enters the component.</p>
<p>This component can be used in two modes:</p>
<ul>
<li>If <i>Counter Property</i> is the name of a local property (does not start with "@"), a new property by that name is created on each data record and given the value of the current count.</li>
<li>If <i>Counter Property</i> is the name of a global property (starts with "@"), then each data record that passes through causes this global record count to be incremented, but values are not added to the passing records.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
17.1.0.119
Local Component Identifier
15
The original path within the database of this component.
Components\Data Access and Manipulation\Data Manipulators\Count and Index Data
Select an Entrez database for the input primary ID:
<ul>
<li>By database property: uses the input database property.</li>
<li>PubMed: biomedical literature citations and abstracts.</li>
<li>PubMed Central: free, full text journal articles.</li>
<li>OMIM: online Mendelian inheritance in man.</li>
<li>Nucleotide: full nucleotide sequence records.</li>
<li>Core Nucleotide: core subset of nucleotide sequence records.</li>
<li>EST: expressed sequence tag records.</li>
<li>GSS: genome survey sequence records.</li>
<li>Protein: sequence database.</li>
<li>Taxonomy: organisms in GenBank.</li>
<li>SNP: single nucleotide polymorphism.</li>
<li>Gene: gene-centered information.</li>
<li>PopSet: population study data sets.</li>
</ul>
<p>
<p>
<b>See Also: </b><a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/sites/gquery')">Entrez search page</a> for detailed database information.
Keep PubMed Abstract HTML Tags
By database property
PubMed
PubMed Central
OMIM
Nucleotide
Core Nucleotide
EST
GSS
Protein
Taxonomy
SNP
Gene
PopSet
<p>To not overload NCBI's systems, database accesses are grouped together in batch requests. Specify the maximum number of primary IDs of each batch request. An exception is <b>PopSet</b>, its Batch Size is always set to 1 internally. Note that accesses to different databases are grouped separately, also there are 3 seconds delay between requests.
SetInvalidDetails(
ParameterName() . ' must be larger than zero');
Parameter() > 0;
500
Read the features and annotations or just the sequence data, description, and accession number for <b>EST</b>, <b>GSS</b>, <b>Nucleotide</b>, <b>Core Nucleotide</b>, or <b>Protein</b> databases.
True
False
Use your NCBI account for high-frequency E-utilities requests.
NCBI Account API Key
NCBI Account Tool
NCBI Account Email
The API key of your NCBI account.
Specifies the software that is producing the request.
The email of your NCBI account.
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
For <b>PubMed</b> database, the abstract field may contain HTML tags like <i> or <sup>. Set this value to True, if you want to keep these tags.
True
False
Entrez EFetch
Protocol AutoLayout
1
0
JOINNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{9579C91D-05FA-4C2F-9D84-125CCB6B9542}
1065 -275
By Ref
1
100
Retreives a record from Entrez databases
<p>
Uses the Entrez EFetch utility to retrieve the Entrez database record from <i>primaryID</i> of the input data record. The primaryID must be used in conjunction with a database. If the input data record has the information in its <i>database</i> property, set <i>Database</i> to <b>By database property</b>. Otherwise, select a suitable database. If the <i>database</i> property is present but not equal to the <i>Database</i> parameter, the input data record is sent out the Fail port.
</p>
<p>The results are processed differently for different databases. For <b>Nucleotide</b>, <b>Core Nucleotide</b>, <b>Protein</b>, <b>EST</b>, and <b>GSS</b>, the database record
is fetched in GenBank format, then loaded by <a href="javascript:DoLink('{034F926A-9B95-45D9-957C-68A2632FDE3A}')">GenBank Sequence Reader</a> and merged into the input data record. The type of the output data record is set to <b>SciTegic.Sequence</b>.
</p>
<p>
If the input data record has no GI number in its <i>primaryID</i>, its <i>accessionNumber</i> will be used as the identifier to fetch the record from NCBI.
</p>
<p>
For other databases, the database record is fetched in XML format, then loaded by <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a> and merged into the input data record, with major information parsed and provided in top-level properties.
</p>
<p>A partial nucleotide sequence can be fetched by specify <i>StartBaseNumber</i> and <i>StopBaseNumber</i> in the input data record. Also specify the input <i>Strand</i> property as <b>Plus</b> or <b>Minus</b> to fetch the desired strand.
</p>
<b>See Also:</b> <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/entrez/query/static/efetch_help.html')">NCBI EFetch utility help</a>
<p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
20.1.0.2156
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Wed Sep 11 11:00:44 2019
Local Component Identifier
1
Subprotocol Made Link Into Copy
0
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
Subprotocol Modified
0
The original path within the database of this component.
Components\Biology\Sequence Analysis\Database Query\Entrez\Entrez EFetch
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allows the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running locally or remotely. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option with care, since it will not increase throughput in all circumstances. Generally speaking, it yields improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
end if;
#parallel;
<p>
A comma-separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (that is, server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be useful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
A comma-separated list of the maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.</br></br>For non-Grid Engine job, the number of jobs will be limited to the value for the maximum number of simultaneous subprotocol jobs as defined in the Admin Console even if this value is higher than the value entered here or '*'.</br></br>For Grid Engine jobs, the number of jobs will be limited to the value for the maximum number of simultaneous subprotocol grid jobs as defined in the Admin Console even if this value is higher than the value entered here or '*'.</br></br>
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available. Note: If no grid engine is configured the subprotocol will still run even if this parameter is set to true.
Parallel Processing Options Server Run On Grid Queue Name
Parallel Processing Options Server Run On Grid Grid Options
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
<p>This is an array of command line options to send to the Grid engine submit command.</p>
<p>If you just include one line, whatever you enter will be sent "as is" to the submission command.</p>
</br>
<b>Example - Just sending options directly to command line - PBS</b>
<p>-l procs=16 -l walltime=08:00:00 -A PROJECTID</p>
</br>
<p>There are a number of parameters that Pipeline Pilot will translate to the correct command line options for whichever supported Grid Engine you are using.</p>
</br>
<b>Example - Supported parameters</b>
<p>Account=<Project/Account name></p>
<p>Maximum Execution Time=<HH:MM:SS> hours, minutes, seconds - LSF ignores seconds</p>
<p>Number of Cores=<16></p>
<p>Parallel Environment=<The Parallel Environment to use> - Only needed for SGE</p>
<p>Other=<Additional Parameters sent directly to submit command line></p>
</br>
<p>Any of these options can be commented out by placing a # in front of them.</p>
</br>
parameter('Parallel Processing Options Server Run On Grid');
Account=
Maximum Execution Time=
Parallel Environment=
Number of Cores=
Other=
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
<p>A comma-delimited list of global properties, each of which is automatically populated with the name of a file in the job's temporary folder, which always has a unique name.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold only the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Separate property names with commas. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>
<p>Any global property not declared in this way is created on the top-level global property list and is accessible from all subprotocols, but breaks the encapsulation of the subprotocol and may collide with another property of the same name.</p>
<p>Therefore, you should always declare a new global property in the <i>DeclareLocal</i> parameter, to minimize the scope of the global property to the subprotocol where it is used.</p>
<p>The example <i>DeclareLocal</i> value below creates 3 global properties in the subprotocol scope. Note that you can initialize the declared global value with a scalar value using the ":=" syntax, as shown. You can surround text string values with quotes to aid clarity.</p>
<pre>
SubP_Counter := 0, SubP_NameHash, SubP_TitleText:="Sample 1"
</pre>
warning
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
no gi, use efetch to get from accession
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
465 -455
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open
a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display
icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them.
For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols
Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see
<a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel
Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut,
so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating
Shortcuts</a>.
</p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
17.2.0.1267
%VERSION 3%
1503 -629 652 264
255 255 0
This shouldn't be needed but leave in place as a backstop in case the accession doesn't conform to expectations.
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Thu Sep 15 15:57:54 2016
Local Component Identifier
5
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
<p>The name of the property in the data record holding the text query.</p>
accessionNumber
Select a Entrez database to search:
<ul>
<li>Assembly: Genome assembly.</li>
<li>BioProject: a collection of biological data related to a single initiative, originating from a single organization or from a consortium.</li>
<li>BioSample: biological source materials used in experimental assays.</li>
<li>BioSystems: biological systems database.</li>
<li>BLAST Database: information for BLAST databases.</li>
<li>Books: online books.</li>
<li>ClinVar: information about genomic variation and its relationship to human health.</li><li>Clone: a database that integrates information about clones and libraries, including sequence data, map positions and distributor information.</li>
<li>Conserved Domains: conserved protein domain database.</li>
<li>Core Nucleotide: core subset of nucleotide sequence records.</li>
<li>dbGaP: genotype and phenotype.</li>
<li>EST: expressed sequence tag records.</li>
<li>Gene: gene-centered information.</li>
<li>Genome: whole genome sequences.</li>
<li>GEO Profiles: expression and molecular abundance profiles.</li>
<li>GEO DataSets: experimental sets of GEO data.</li>
<li>GSS: genome survey sequence records.</li>
<li>GTR: a central location for voluntary submission of genetic test information by providers.</li>
<li>HomoloGene: eukaryotic homology groups.</li>
<li>MedGen: organized information related to human medical genetics, such as attributes of conditions with a genetic contribution.</li>
<li>MeSH: detailed information about NLM's controlled vocabulary.</li>
<li>NCBI Web Site: NCBI site and ftp search.</li>
<li>NLM Catalog: catalog of books, journals, and audiovisuals in the NLM collections.</li>
<li>Nucleotide: full nucleotide sequence records.</li>
<li>OMIM: online Mendelian inheritance in man.</li>
<li>PopSet: population study data sets.</li>
<li>Probe: sequence-specific reagents.</li>
<li>Protein: sequence database.</li>
<li>Protein Clusters: a collection of related protein sequences.</li>
<li>PubChem BioAssay: bioactivity screens of chemical substances.</li>
<li>PubChem Compound: unique small molecule chemical structures.</li>
<li>PubChem Substance: deposited chemical substance records.</li>
<li>PubMed: biomedical literature citations and abstracts.</li>
<li>PubMed Central: free, full text journal articles.</li>
<li>PubMed Health: specialized in clinical effectiveness research.</li>
<li>SNP: single nucleotide polymorphism.</li>
<li>Sparcle: Subfamily Protein Architecture Labeling Engine.</li>
<li>SRA: next generation sequencing data.</li>
<li>Structure: three-dimensional macromolecular structures.</li>
<li>Taxonomy: organisms in GenBank.</li>
<li>UniGene: gene-oriented clusters of transcript sequences.</li>
</ul>
<p>
<p>
<b>See Also:</b> <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/sites/gquery')">Entrez search page</a> for detailed database information.
ID Type
Assembly
BioProject
BioSample
BioSystems
BLAST Database
Books
ClinVar
Clone
Conserved Domains
Core Nucleotide
dbGaP
EST
Gene
Genome
GEO DataSets
GEO Profiles
GSS
GTR
HomoloGene
MedGen
MeSH
NCBI Web Site
NLM Catalog
Nucleotide
OMIM
PopSet
Probe
Protein
Protein Clusters
PubChem BioAssay
PubChem Compound
PubChem Substance
PubMed
PubMed Central
PubMed Health
SNP
Sparcle
SRA
Structure
Taxonomy
UniGene
ID Type
UsingParent
For <b>Nucleotide</b>, <b>Core Nucleotide</b>, <b>EST</b>, <b>GSS</b>, <b>Protein</b> databases, Specifies whether to return accession.version or GI number. The output data records will use <i>accessionNumber</i> for <b>Accession.Version</b> but <i>primaryID</i> for <b>GI</b>.
SetDisabledDetails(
'To enable '.ParameterName().', set Database to "Nucleotide", "Core Nucleotide", "EST", "GSS", or "Protein"');
Parameter('Database') eq 'Nucleotide' or
Parameter('Database') eq 'Core Nucleotide' or
Parameter('Database') eq 'EST' or
Parameter('Database') eq 'GSS' or
Parameter('Database') eq 'Protein';
Accession.Version
GI
Select a field to search or select <b>All Fields [all]</b> to search all fields. The legal values are dependent on <i>Database</i>.
resize(lvals, 1);
lvals[1] := "All Fields [all]";
SetLegalValues(lvals);
#database := Parameter('Database');
if (#database eq 'Assembly') then
resize(lvals, 41);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "Assembly Accession [asac]";
lvals[4] := "Assembly Level [aslv]";
lvals[5] := "Taxonomy ID [txid]";
lvals[6] := "Organism [orgn]";
lvals[7] := "RefSeq ID [ruid]";
lvals[8] := "GenBank ID [guid]";
lvals[9] := "All Uids [uids]";
lvals[10] := "BioProject IDs and Accessions [proj]";
lvals[11] := "BioSample [samp]";
lvals[12] := "Assembly Name [name]";
lvals[13] := "All Names [alln]";
lvals[14] := "Description [desc]";
lvals[15] := "Coverage [cov]";
lvals[16] := "Assembly Type [type]";
lvals[17] := "Total Sequence Length in Mbp [len]";
lvals[18] := "Chromosome Count [repl]";
lvals[19] := "Placed Scaffolds Count [plac]";
lvals[20] := "Unlocalized Scaffolds Count [unlo]";
lvals[21] := "Unplaced Scaffolds Count [unpl]";
lvals[22] := "Contig N50 [cn50]";
lvals[23] := "Scaffold N50 [sn50]";
lvals[24] := "Contig L50 [cl50]";
lvals[25] := "Scaffold L50 [sl50]";
lvals[26] := "Contig Count [cntg]";
lvals[27] := "Ungapped Length in Mbp [ungl]";
lvals[28] := "Properties [prop]";
lvals[29] := "Submitter Organization [subo]";
lvals[30] := "Infraspecific Name [infr]";
lvals[31] := "Isolate [isol]";
lvals[32] := "Sex [sex]";
lvals[33] := "Assembly Method [asmm]";
lvals[34] := "Genome Coverage [gcov]";
lvals[35] := "Sequencing Technology [tech]";
lvals[36] := "Expected Final Version [exfv]";
lvals[37] := "Reference Guided Assembly [rgas]";
lvals[38] := "Single Cell Amplification [scam]";
lvals[39] := "RefSeq Category [rcat]";
lvals[40] := "From Type Material [ftyp]";
lvals[41] := "Excluded from RefSeq [nfrs]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioProject') then
resize(lvals, 24);
lvals[1] := "All Fields [all]";
lvals[2] := "Organism [orgn]";
lvals[3] := "Project Accession [prja]";
lvals[4] := "Project Type [type]";
lvals[5] := "Project Subtype [stpe]";
lvals[6] := "Title [titl]";
lvals[7] := "Submitter Organization [cen]";
lvals[8] := "Replicon accession [accn]";
lvals[9] := "Replicon type [rtyp]";
lvals[10] := "Locus Tag Prefix [ltp]";
lvals[11] := "Description [word]";
lvals[12] := "Keyword [kwrd]";
lvals[13] := "Properties [prop]";
lvals[14] := "Project Data Type [dtpe]";
lvals[15] := "Grant ID [grnt]";
lvals[16] := "Funding Agency [fund]";
lvals[17] := "PMID [pmid]";
lvals[18] := "ProjectID [pid]";
lvals[19] := "Relevance [relv]";
lvals[20] := "Assembly name [anme]";
lvals[21] := "WGS Accession [wgsa]";
lvals[22] := "Assembly Accession [aacc]";
lvals[23] := "Attribute Name [atnm]";
lvals[24] := "Attribute [attr]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioSample') then
resize(lvals, 10);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "Title [titl]";
lvals[4] := "Properties [prop]";
lvals[5] := "Text Word [word]";
lvals[6] := "Organism [orgn]";
lvals[7] := "Author [auth]";
lvals[8] := "Attribute Name [atnm]";
lvals[9] := "Attribute [attr]";
lvals[10] := "Submitter Organization [cen]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioSystems') then
resize(lvals, 23);
lvals[1] := "All Fields [all]";
lvals[2] := "SourceName [src]";
lvals[3] := "SourceID [srid]";
lvals[4] := "BioSystemType [type]";
lvals[5] := "SourceAccession [sacc]";
lvals[6] := "Title [titl]";
lvals[7] := "Description [desc]";
lvals[8] := "Comments [com]";
lvals[9] := "Organism [orgn]";
lvals[10] := "ProteinName [pn]";
lvals[11] := "ChemicalName [cn]";
lvals[12] := "SidExternalID [seid]";
lvals[13] := "GeneName [gn]";
lvals[14] := "GeneExternalID [geid]";
lvals[15] := "ProteinID [pid]";
lvals[16] := "CID [cid]";
lvals[17] := "GeneID [gid]";
lvals[18] := "SID [sid]";
lvals[19] := "Accession [accn]";
lvals[20] := "SIDCount [sct]";
lvals[21] := "CIDCount [cct]";
lvals[22] := "GeneCount [gct]";
lvals[23] := "ProteinCount [pct]";
SetLegalValues(lvals);
end if;
if (#database eq 'Books') then
resize(lvals, 26);
lvals[1] := "All Fields [all]";
lvals[2] := "Author [auth]";
lvals[3] := "Corporate Author [ca]";
lvals[4] := "Full Author Name [fa]";
lvals[5] := "Full Editor Name [fe]";
lvals[6] := "Title [titl]";
lvals[7] := "Type [type]";
lvals[8] := "Full Text [stxt]";
lvals[9] := "Concept Phrases [conp]";
lvals[10] := "Book [book]";
lvals[11] := "PMID [pmid]";
lvals[12] := "RefPMID [rmid]";
lvals[13] := "Rid [rid]";
lvals[14] := "Publisher [pubn]";
lvals[15] := "ISBN [isbn]";
lvals[16] := "Attribute [attr]";
lvals[17] := "Editor [edit]";
lvals[18] := "Subject [sub]";
lvals[19] := "Resource Type [rt]";
lvals[20] := "Accession ID [aid]";
lvals[21] := "Book Accession ID [baci]";
lvals[22] := "Chapter Accession ID [chid]";
lvals[23] := "Disease [dn]";
lvals[24] := "Gene Name [gs]";
lvals[25] := "Protein Name [pn]";
lvals[26] := "Grant Number [grnt]";
SetLegalValues(lvals);
end if;
if (#database eq 'ClinVar') then
resize(lvals, 34);
lvals[1] := "All Fields [all]";
lvals[2] := "Name of the ClinVar record [titl]";
lvals[3] := "Text Word [word]";
lvals[4] := "Organism [orgn]";
lvals[5] := "Chromosome [chr]";
lvals[6] := "Gene Name [gene]";
lvals[7] := "MIM [mim]";
lvals[8] := "Disease/Phenotype [dis]";
lvals[9] := "ClinVar accession [accn]";
lvals[10] := "External allele ID [vrid]";
lvals[11] := "Trait identifier [trid]";
lvals[12] := "Properties [prop]";
lvals[13] := "PubMed ID [pmid]";
lvals[14] := "Gene ID [gid]";
lvals[15] := "Taxonomy ID [tid]";
lvals[16] := "Base Position [cpos]";
lvals[17] := "Gene Full Name [gfn]";
lvals[18] := "Submitter [sbm]";
lvals[19] := "Variant name [vrnm]";
lvals[20] := "Type of variation [vrtp]";
lvals[21] := "Molecular consequence [mcns]";
lvals[22] := "Review status [rvst]";
lvals[23] := "AlleleID [alid]";
lvals[24] := "Origin [orig]";
lvals[25] := "Submitter Batch [sid]";
lvals[26] := "Cytogenetic band [cyt]";
lvals[27] := "Accession for a test registered in GTR [gtrt]";
lvals[28] := "Study Name [stnm]";
lvals[29] := "Complexity [cmpl]";
lvals[30] := "Base Position for Assembly GRCh37 [c37]";
lvals[31] := "Length of the variant [vlen]";
lvals[32] := "Nucleotide/Protein Accession [accn]";
lvals[33] := "HGNC identifier for human gene [hgnc]";
lvals[34] := "Variation ID [vid]";
SetLegalValues(lvals);
end if;
if (#database eq 'Clone') then
resize(lvals, 49);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [acc]";
lvals[3] := "Accession Version [accv]";
lvals[4] := "Alternate Library Abbreviation [alab]";
lvals[5] := "Alternate Library Name [aln]";
lvals[6] := "Assembly Accession [assa]";
lvals[7] := "Assembly Name [assn]";
lvals[8] := "Breed [bree]";
lvals[9] := "Cell Line [cell]";
lvals[10] := "Cell Type [celt]";
lvals[11] := "Chromosome Accession [chra]";
lvals[12] := "Chromosome Start [chrb]";
lvals[13] := "Chromosome Stop [chre]";
lvals[14] := "Clone Name [cln]";
lvals[15] := "Clone Name Alias [cla]";
lvals[16] := "Cultivar [cult]";
lvals[17] := "Cultivar Accession [cuac]";
lvals[18] := "Distributor [dist]";
lvals[19] := "Development Stage [dest]";
lvals[20] := "Gene Name [gene]";
lvals[21] := "Gene ID [gnid]";
lvals[22] := "Gene Description [gdsc]";
lvals[23] := "GI [gi]";
lvals[24] := "Isolate [isol]";
lvals[25] := "Library Abbreviation [lbr]";
lvals[26] := "Library ID [lid]";
lvals[27] := "Library Name [lib]";
lvals[28] := "Library Type [libt]";
lvals[29] := "Object type [ot]";
lvals[30] := "Organ [orga]";
lvals[31] := "Organism [org]";
lvals[32] := "Placed [plcd]";
lvals[33] := "Placement Method [plmt]";
lvals[34] := "Placement Confidence [plcn]";
lvals[35] := "Population [popu]";
lvals[36] := "Properties [prop]";
lvals[37] := "Genome Project Id [pid]";
lvals[38] := "PMID [pmid]";
lvals[39] := "Scaffold Accession [scfa]";
lvals[40] := "Scaffold Start [scfb]";
lvals[41] := "Scaffold Stop [scfe]";
lvals[42] := "Strain [stra]";
lvals[43] := "STS [sts]";
lvals[44] := "Taxonomy ID [txid]";
lvals[45] := "TI [ti]";
lvals[46] := "Tissue [tiss]";
lvals[47] := "Vector [vn]";
lvals[48] := "Vector Type [vt]";
lvals[49] := "BAC Resource [ba]";
SetLegalValues(lvals);
end if;
if (#database eq 'Conserved Domains') then
resize(lvals, 12);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "Database [db]";
lvals[4] := "Title [titl]";
lvals[5] := "Subtitle [sttl]";
lvals[6] := "Text Word [word]";
lvals[7] := "Organism [orgn]";
lvals[8] := "PssmLength [plen]";
lvals[9] := "Alternative Accession [aacn]";
lvals[10] := "Structure Representative [strp]";
lvals[11] := "The description of sites [sd]";
lvals[12] := "Number of Sites [ns]";
SetLegalValues(lvals);
end if;
if (#database eq 'Core Nucleotide') then
resize(lvals, 30);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Sequence Length [slen]";
lvals[21] := "Feature key [fkey]";
lvals[22] := "Primary Organism [porg]";
lvals[23] := "Component Accession [comp]";
lvals[24] := "Assembly [assm]";
lvals[25] := "Division [div]";
lvals[26] := "Strain [strn]";
lvals[27] := "Isolate [isol]";
lvals[28] := "Cultivar [cult]";
lvals[29] := "Breed [brd]";
lvals[30] := "BioSample [bios]";
SetLegalValues(lvals);
end if;
if (#database eq 'EST') then
resize(lvals, 32);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Sequence Length [slen]";
lvals[21] := "Feature key [fkey]";
lvals[22] := "EST id [id]";
lvals[23] := "EST Name [name]";
lvals[24] := "Clone ID [clon]";
lvals[25] := "Library Name [lib]";
lvals[26] := "Submitter Name [subm]";
lvals[27] := "Citation Title [cit]";
lvals[28] := "Strain [strn]";
lvals[29] := "Isolate [isol]";
lvals[30] := "Cultivar [cult]";
lvals[31] := "Breed [brd]";
lvals[32] := "BioSample [bios]";
SetLegalValues(lvals);
end if;
if (#database eq 'GEO DataSets') then
resize(lvals, 27);
lvals[1] := "All Fields [all]";
lvals[2] := "Organism [orgn]";
lvals[3] := "GEO Accession [accn]";
lvals[4] := "Title [titl]";
lvals[5] := "Description [desc]";
lvals[6] := "Supplementary Files [sfil]";
lvals[7] := "Entry Type [etyp]";
lvals[8] := "Sample Type [styp]";
lvals[9] := "Sample Value Type [vtyp]";
lvals[10] := "Platform Technology Type [ptyp]";
lvals[11] := "DataSet Type [gtyp]";
lvals[12] := "Number of Samples [nsam]";
lvals[13] := "Sample Source [src]";
lvals[14] := "Author [auth]";
lvals[15] := "Submitter Institute [inst]";
lvals[16] := "Number of Platform Probes [npro]";
lvals[17] := "Subset Variable Type [sstp]";
lvals[18] := "Subset Description [ssde]";
lvals[19] := "Reporter Identifier [geid]";
lvals[20] := "Tag Length [tagl]";
lvals[21] := "Related Series [rgse]";
lvals[22] := "Related Platform [rgpl]";
lvals[23] := "MeSH Terms [mesh]";
lvals[24] := "Project [proj]";
lvals[25] := "Attribute Name [atnm]";
lvals[26] := "Attribute [attr]";
lvals[27] := "Properties [prop]";
SetLegalValues(lvals);
end if;
if (#database eq 'GEO Profiles') then
resize(lvals, 24);
lvals[1] := "All Fields [all]";
lvals[2] := "Organism [orgn]";
lvals[3] := "GEO Accession [accn]";
lvals[4] := "GDS Text [gdst]";
lvals[5] := "GEO Description/Title Text [geot]";
lvals[6] := "Platform Reporter Type [rtyp]";
lvals[7] := "DataSet Type [gtyp]";
lvals[8] := "Sample Value Type [vtyp]";
lvals[9] := "Number of Samples [nsam]";
lvals[10] := "Sample Source [src]";
lvals[11] := "ID_REF [id]";
lvals[12] := "Reporter Identifier [name]";
lvals[13] := "Gene Symbol [symb]";
lvals[14] := "Gene Description [gdsc]";
lvals[15] := "Ranked Standard Deviation [rstd]";
lvals[16] := "Max Value Rank [rmax]";
lvals[17] := "Min Value Rank [rmin]";
lvals[18] := "Flag Information [finf]";
lvals[19] := "Flag Type [ftyp]";
lvals[20] := "GI [gi]";
lvals[21] := "Annotation Type [atyp]";
lvals[22] := "Gene Ontology [go]";
lvals[23] := "Chromosome [chr]";
lvals[24] := "Base Position [cpos]";
SetLegalValues(lvals);
end if;
if (#database eq 'GSS') then
resize(lvals, 33);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Sequence Length [slen]";
lvals[21] := "Feature key [fkey]";
lvals[22] := "GSS id [id]";
lvals[23] := "GSS Name [name]";
lvals[24] := "Clone ID [clon]";
lvals[25] := "Library Name [lib]";
lvals[26] := "Submitter Name [subm]";
lvals[27] := "Citation Title [cit]";
lvals[28] := "Library Class [lc]";
lvals[29] := "Strain [strn]";
lvals[30] := "Isolate [isol]";
lvals[31] := "Cultivar [cult]";
lvals[32] := "Breed [brd]";
lvals[33] := "BioSample [bios]";
SetLegalValues(lvals);
end if;
if (#database eq 'GTR') then
resize(lvals, 47);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Text Word 1 [wrd1]";
lvals[4] := "Properties [prop]";
lvals[5] := "Organization names [org]";
lvals[6] := "Organization City [city]";
lvals[7] := "Organization State [state]";
lvals[8] := "Organization Country [country]";
lvals[9] := "Organization postcode [postcode]";
lvals[10] := "Organization Director(s) [director]";
lvals[11] := "Organization Staff [staff]";
lvals[12] := "Lab Service name [ls]";
lvals[13] := "Accession for GTR test [gtracc]";
lvals[14] := "name for this test [name]";
lvals[15] := "Alternate name [alt]";
lvals[16] := "disease name for this test [testdis]";
lvals[17] := "Specimen options for test [specimen]";
lvals[18] := "title of this clinvar assertion [titl]";
lvals[19] := "Accession for clinvar assertion [cvacc]";
lvals[20] := "Disease BioConcepts concept id [dcui]";
lvals[21] := "Name of Disease [disname]";
lvals[22] := "Gene BioConcepts concept id [gcui]";
lvals[23] := "Gene Symbol [symb]";
lvals[24] := "Name of Gene [genename]";
lvals[25] := "MIM number for the Gene [genemim]";
lvals[26] := "Name of Protein [protname]";
lvals[27] := "Name of Method [mtod]";
lvals[28] := "name for method category [mcat]";
lvals[29] := "name for method top category [tcat]";
lvals[30] := "laboratory certification [lcrt]";
lvals[31] := "Country ISO code [cid]";
lvals[32] := "State ISO code [sid]";
lvals[33] := "Top Method Category ID [tcid]";
lvals[34] := "Method Category ID [ctid]";
lvals[35] := "Test Method ID [mtid]";
lvals[36] := "Disease BioConcepts Entrez ID [did]";
lvals[37] := "Organism [orgn]";
lvals[38] := "Target Population option for the test [tarpop]";
lvals[39] := "Clinical Utility option for the test [clnutl]";
lvals[40] := "Clinical Validity option for the test [clnval]";
lvals[41] := "Number of genes [genenum]";
lvals[42] := "Pharmacogenetic response condition [pharma]";
lvals[43] := "Specimen Type ID [stid]";
lvals[44] := "Test target [testtarget]";
lvals[45] := "Analyte [alyt]";
lvals[46] := "Chromosomal region [chrg]";
lvals[47] := "Test gene [testgene]";
SetLegalValues(lvals);
end if;
if (#database eq 'Gene') then
resize(lvals, 30);
lvals[1] := "All Fields [all]";
lvals[2] := "Gene/Protein Name [titl]";
lvals[3] := "Text Word [word]";
lvals[4] := "Organism [orgn]";
lvals[5] := "Chromosome [chr]";
lvals[6] := "Default Map Location [mv]";
lvals[7] := "Gene Name [gene]";
lvals[8] := "EC/RN Number [ecno]";
lvals[9] := "MIM ID [mim]";
lvals[10] := "Disease/Phenotype [dis]";
lvals[11] := "Nucleotide/Protein Accession [accn]";
lvals[12] := "UniGene Cluster Number [ugen]";
lvals[13] := "Properties [prop]";
lvals[14] := "Nucleotide Accession [ncac]";
lvals[15] := "Nucleotide UID [nuid]";
lvals[16] := "Protein Accession [pacc]";
lvals[17] := "Protein UID [puid]";
lvals[18] := "PubMed ID [pmid]";
lvals[19] := "Taxonomy ID [tid]";
lvals[20] := "Gene Ontology [go]";
lvals[21] := "Domain Name [dom]";
lvals[22] := "Base Position [cpos]";
lvals[23] := "Gene Full Name [gfn]";
lvals[24] := "Protein Full Name [pfn]";
lvals[25] := "Gene Length [gl]";
lvals[26] := "Exon Count [xc]";
lvals[27] := "Group [grp]";
lvals[28] := "Preferred Symbol [pref]";
lvals[29] := "Assembly Accession [aacc]";
lvals[30] := "Assembly Name [asm]";
SetLegalValues(lvals);
end if;
if (#database eq 'Genome') then
resize(lvals, 27);
lvals[1] := "All Fields [all]";
lvals[2] := "Organism [orgn]";
lvals[3] := "ProjectID [pid]";
lvals[4] := "Project Accession [prja]";
lvals[5] := "Project Type [prjt]";
lvals[6] := "Title [dfln]";
lvals[7] := "Genome description [dscr]";
lvals[8] := "Status [stat]";
lvals[9] := "AssemblyID [aid]";
lvals[10] := "Assembly Accession [aacc]";
lvals[11] := "Assembly Name [anam]";
lvals[12] := "Replicon GI [gi]";
lvals[13] := "Replicon accession [accn]";
lvals[14] := "Replicon name [rnam]";
lvals[15] := "Protein Accession [pacc]";
lvals[16] := "Protein Name [prot]";
lvals[17] := "Protein GI [pgi]";
lvals[18] := "GeneID [gnid]";
lvals[19] := "Gene Name [gene]";
lvals[20] := "Locus Tag [ltag]";
lvals[21] := "WGS prefix [wgsp]";
lvals[22] := "PubMed ID [pmid]";
lvals[23] := "biological properties [biop]";
lvals[24] := "ProtClust ID [pcid]";
lvals[25] := "Properties [prop]";
lvals[26] := "Strain [strn]";
lvals[27] := "Host [host]";
SetLegalValues(lvals);
end if;
if (#database eq 'HomoloGene') then
resize(lvals, 15);
lvals[1] := "All Fields [all]";
lvals[2] := "Title [titl]";
lvals[3] := "Text Word [word]";
lvals[4] := "Properties [prop]";
lvals[5] := "Organism [orgn]";
lvals[6] := "Gene ID [gnid]";
lvals[7] := "Gene Name [gene]";
lvals[8] := "Gene Description [gdsc]";
lvals[9] := "Protein UID [puid]";
lvals[10] := "Protein Accession [prac]";
lvals[11] := "Nucleotide UID [nuid]";
lvals[12] := "Nucleotide Accession [ncac]";
lvals[13] := "UniGene ID [ugid]";
lvals[14] := "Ancestor [ancs]";
lvals[15] := "Domain Name [dom]";
SetLegalValues(lvals);
end if;
if (#database eq 'MeSH') then
resize(lvals, 8);
lvals[1] := "All Fields [all]";
lvals[2] := "MeSH Terms [mesh]";
lvals[3] := "Substance Name [subs]";
lvals[4] := "Text Word [word]";
lvals[5] := "Scope Note [note]";
lvals[6] := "Registry Number [reg]";
lvals[7] := "Record Type [type]";
lvals[8] := "MeSH Unique ID [mhui]";
SetLegalValues(lvals);
end if;
if (#database eq 'MedGen') then
resize(lvals, 19);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "Title [titl]";
lvals[4] := "Properties [prop]";
lvals[5] := "Text Word [word]";
lvals[6] := "Definition [defn]";
lvals[7] := "Vocabulary [vocb]";
lvals[8] := "ExactTitle [xtit]";
lvals[9] := "Source ID [code]";
lvals[10] := "Chromosome [chr]";
lvals[11] := "Clinical Features [clin]";
lvals[12] := "Gene Name [gene]";
lvals[13] := "Base Position [cpos]";
lvals[14] := "MIM ID [oid]";
lvals[15] := "Gene Full Name [gfn]";
lvals[16] := "Keyword [kwd]";
lvals[17] := "Mode Of Inheritance [minh]";
lvals[18] := "SNOMED CT CUI [snm]";
lvals[19] := "Guideline title [gtit]";
SetLegalValues(lvals);
end if;
if (#database eq 'NCBI Web Site') then
resize(lvals, 11);
lvals[1] := "All Fields [all]";
lvals[2] := "Title [titl]";
lvals[3] := "Keyword [kywd]";
lvals[4] := "Description [desc]";
lvals[5] := "Text Word [word]";
lvals[6] := "Category [cat]";
lvals[7] := "Homepage Title [home]";
lvals[8] := "Homepage URL [hurl]";
lvals[9] := "URL [url]";
lvals[10] := "META Name [mnam]";
lvals[11] := "META Content [mcon]";
SetLegalValues(lvals);
end if;
if (#database eq 'NLM Catalog') then
resize(lvals, 37);
lvals[1] := "All Fields [all]";
lvals[2] := "Abstract/Index Tags [itag]";
lvals[3] := "Author [auth]";
lvals[4] := "Author Full Name [full]";
lvals[5] := "Call Number [call]";
lvals[6] := "Corporate/Conference Name [cnam]";
lvals[7] := "Item Type [item]";
lvals[8] := "Journal [jour]";
lvals[9] := "Language [lang]";
lvals[10] := "Resource Type [rtyp]";
lvals[11] := "MeSH Terms [mesh]";
lvals[12] := "MeSH Major Topic [majr]";
lvals[13] := "MeSH Subheading [subh]";
lvals[14] := "Olio [olio]";
lvals[15] := "Other Number [othr]";
lvals[16] := "Other Term [otrm]";
lvals[17] := "Personal Name as Subject [pers]";
lvals[18] := "Personal Full Name as Subject [fper]";
lvals[19] := "Country of Publication [cnty]";
lvals[20] := "Publication Year [pdat]";
lvals[21] := "Publication Status [psta]";
lvals[22] := "Publication Type [ptyp]";
lvals[23] := "Publisher [publ]";
lvals[24] := "Series [seri]";
lvals[25] := "Title [titl]";
lvals[26] := "URL [url]";
lvals[27] := "NLM Unique ID [nid]";
lvals[28] := "Authority Information [ai]";
lvals[29] := "AIID [aiid]";
lvals[30] := "Broad Subject Term [st]";
lvals[31] := "Current Format Status [cfs]";
lvals[32] := "Indexing Subset [xs]";
lvals[33] := "ISO Abbreviation [iso]";
lvals[34] := "Publication Start Year [syr]";
lvals[35] := "Publication End Year [eyr]";
lvals[36] := "ISSN [is]";
lvals[37] := "NLM Title Abbreviation [mabr]";
SetLegalValues(lvals);
end if;
if (#database eq 'Nucleotide') then
resize(lvals, 30);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Sequence Length [slen]";
lvals[21] := "Feature key [fkey]";
lvals[22] := "Primary Organism [porg]";
lvals[23] := "Component Accession [comp]";
lvals[24] := "Assembly [assm]";
lvals[25] := "Division [div]";
lvals[26] := "Strain [strn]";
lvals[27] := "Isolate [isol]";
lvals[28] := "Cultivar [cult]";
lvals[29] := "Breed [brd]";
lvals[30] := "BioSample [bios]";
SetLegalValues(lvals);
end if;
if (#database eq 'OMIM') then
resize(lvals, 16);
lvals[1] := "All Fields [all]";
lvals[2] := "Title [titl]";
lvals[3] := "Text Word [word]";
lvals[4] := "Contributor [auth]";
lvals[5] := "Clinical Synopsis [clin]";
lvals[6] := "Allelic Variant [alvr]";
lvals[7] := "Reference [refr]";
lvals[8] := "Gene Map [gmap]";
lvals[9] := "Gene Map Disorder [dsdr]";
lvals[10] := "Gene Name [gene]";
lvals[11] := "EC/RN Number [ecno]";
lvals[12] := "Chromosome [chr]";
lvals[13] := "Editor [edtr]";
lvals[14] := "Properties [prop]";
lvals[15] := "Clinical Synopsis Key [csk]";
lvals[16] := "Clinical Synopsis Editor [csed]";
SetLegalValues(lvals);
end if;
if (#database eq 'PopSet') then
resize(lvals, 26);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Feature key [fkey]";
lvals[21] := "Protein Count [pcnt]";
lvals[22] := "Nucleotide Count [ncnt]";
lvals[23] := "Strain [strn]";
lvals[24] := "Isolate [isol]";
lvals[25] := "Cultivar [cult]";
lvals[26] := "Breed [brd]";
SetLegalValues(lvals);
end if;
if (#database eq 'Probe') then
resize(lvals, 20);
lvals[1] := "All Fields [all]";
lvals[2] := "Author [auth]";
lvals[3] := "Distributor [dist]";
lvals[4] := "Gene Name [gene]";
lvals[5] := "Gene ID [gnid]";
lvals[6] := "Application [kywd]";
lvals[7] := "Organism [orgn]";
lvals[8] := "Probe Name [prnm]";
lvals[9] := "Probe Type [prty]";
lvals[10] := "Properties [prop]";
lvals[11] := "Text Word [word]";
lvals[12] := "Title [titl]";
lvals[13] := "Caption [capt]";
lvals[14] := "Platform Name [coll]";
lvals[15] := "Sequence [seq]";
lvals[16] := "Sequence accession [accn]";
lvals[17] := "Pubmed ID [pmid]";
lvals[18] := "Validation [val]";
lvals[19] := "UniSTS ID [usts]";
lvals[20] := "Probeset UID [pset]";
SetLegalValues(lvals);
end if;
if (#database eq 'Protein') then
resize(lvals, 29);
lvals[1] := "All Fields [all]";
lvals[2] := "Text Word [word]";
lvals[3] := "Title [titl]";
lvals[4] := "Keyword [kywd]";
lvals[5] := "Author [auth]";
lvals[6] := "Journal [jour]";
lvals[7] := "Volume [vol]";
lvals[8] := "Issue [iss]";
lvals[9] := "Page Number [page]";
lvals[10] := "Organism [orgn]";
lvals[11] := "Accession [accn]";
lvals[12] := "Primary Accession [pacc]";
lvals[13] := "Gene Name [gene]";
lvals[14] := "Protein Name [prot]";
lvals[15] := "EC/RN Number [ecno]";
lvals[16] := "Substance Name [subs]";
lvals[17] := "Properties [prop]";
lvals[18] := "SeqID String [sqid]";
lvals[19] := "BioProject [gprj]";
lvals[20] := "Sequence Length [slen]";
lvals[21] := "Molecular Weight [mlwt]";
lvals[22] := "Feature key [fkey]";
lvals[23] := "Primary Organism [porg]";
lvals[24] := "Assembly [assm]";
lvals[25] := "Division [div]";
lvals[26] := "Strain [strn]";
lvals[27] := "Isolate [isol]";
lvals[28] := "Cultivar [cult]";
lvals[29] := "Breed [brd]";
SetLegalValues(lvals);
end if;
if (#database eq 'Protein Clusters') then
resize(lvals, 27);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "PubMed ID [pmid]";
lvals[4] := "Title [titl]";
lvals[5] := "Gene Name [gene]";
lvals[6] := "Gene Synonym [gsyn]";
lvals[7] := "COG [cog]";
lvals[8] := "EC/RN Number [ecno]";
lvals[9] := "HAMAP [hmap]";
lvals[10] := "KO [ko]";
lvals[11] := "Protein Name [prot]";
lvals[12] := "Protein Accession [pacc]";
lvals[13] := "Locus Tag [ltag]";
lvals[14] := "Sequence Length [slen]";
lvals[15] := "Organism [orgn]";
lvals[16] := "Taxonomy ID [txid]";
lvals[17] := "Size [size]";
lvals[18] := "Domain Name [dom]";
lvals[19] := "Domains [doms]";
lvals[20] := "Protein GI [puid]";
lvals[21] := "Paralogs [para]";
lvals[22] := "COG group [cogg]";
lvals[23] := "Average Length [avgl]";
lvals[24] := "Properties [prop]";
lvals[25] := "Total Publications [tpub]";
lvals[26] := "SwissProt Accession [spcn]";
lvals[27] := "Conserved In [cons]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem BioAssay') then
resize(lvals, 36);
lvals[1] := "All Fields [all]";
lvals[2] := "Assay Name [anam]";
lvals[3] := "Assay Description [ades]";
lvals[4] := "Assay Protocol [aprl]";
lvals[5] := "Categorized Comment [ccmt]";
lvals[6] := "Assay Comment [acmt]";
lvals[7] := "Assay Source ID [asrd]";
lvals[8] := "Activity Outcome Method [acmd]";
lvals[9] := "Source Name [snme]";
lvals[10] := "Current Source Name [csnm]";
lvals[11] := "Journal Name [jnam]";
lvals[12] := "Active Sid Count [ac]";
lvals[13] := "Total Sid Count [tsc]";
lvals[14] := "Target Count [tcnt]";
lvals[15] := "Probe Cid Count [pcc]";
lvals[16] := "Probe Sid Count [psc]";
lvals[17] := "Nucleic Acid Reagent ID [nard]";
lvals[18] := "Protein Target Name [ptn]";
lvals[19] := "Gene Symbol [gsym]";
lvals[20] := "Categorized Comment Title [cct]";
lvals[21] := "Protein Target GI [ptgi]";
lvals[22] := "BioSystems ID [bsid]";
lvals[23] := "Pig GI [pigi]";
lvals[24] := "RNA Target GI [rtgi]";
lvals[25] := "Synonym Tested [synt]";
lvals[26] := "Source Category [srcc]";
lvals[27] := "Taxonomy Name [txnm]";
lvals[28] := "Grant Number [grn]";
lvals[29] := "Number of Sids With Activity Concentration micromolar [nsam]";
lvals[30] := "Number of Sids With Activity Concentration nanomolar [nsan]";
lvals[31] := "Organism [orgn]";
lvals[32] := "Assay Project [aprj]";
lvals[33] := "Cell Line [cell]";
lvals[34] := "GenBank Accession [gbac]";
lvals[35] := "UniProt Accession [upac]";
lvals[36] := "Detection Method [dtmd]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem Compound') then
resize(lvals, 39);
lvals[1] := "All Fields [all]";
lvals[2] := "CompoundID [uid]";
lvals[3] := "SourceName [src]";
lvals[4] := "SourceCategory [srcc]";
lvals[5] := "InChI [inch]";
lvals[6] := "InChIKey [ikey]";
lvals[7] := "CompleteSynonym [csyn]";
lvals[8] := "Synonym [syno]";
lvals[9] := "MeSHTerm [msht]";
lvals[10] := "PharmAction [phma]";
lvals[11] := "Element [elmt]";
lvals[12] := "MolecularWeight [mw]";
lvals[13] := "TotalFormalCharge [tfc]";
lvals[14] := "IUPACName [upac]";
lvals[15] := "XLogP [xlgp]";
lvals[16] := "Complexity [cplx]";
lvals[17] := "RotatableBondCount [rbc]";
lvals[18] := "HydrogenBondDonorCount [hbdc]";
lvals[19] := "HydrogenBondAcceptorCount [hbac]";
lvals[20] := "HeavyAtomCount [hac]";
lvals[21] := "AtomChiralCount [acc]";
lvals[22] := "AtomChiralDefCount [acdc]";
lvals[23] := "AtomChiralUndefCount [acuc]";
lvals[24] := "BondChiralCount [bcc]";
lvals[25] := "BondChiralDefCount [bcdc]";
lvals[26] := "BondChiralUndefCount [bcuc]";
lvals[27] := "IsotopeAtomCount [iac]";
lvals[28] := "CovalentUnitCount [cuc]";
lvals[29] := "ActiveAidCount [ac]";
lvals[30] := "TotalAidCount [tac]";
lvals[31] := "SubstanceID [sid]";
lvals[32] := "TPSA [tpsa]";
lvals[33] := "ExactMass [emas]";
lvals[34] := "MonoisotopicMass [mmas]";
lvals[35] := "PharmActionID [paid]";
lvals[36] := "StructureID [stid]";
lvals[37] := "DepositorCompleteSynonym [dcsy]";
lvals[38] := "DepositorSynonym [dsyn]";
lvals[39] := "DepositorCount [dcnt]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem Substance') then
resize(lvals, 15);
lvals[1] := "All Fields [all]";
lvals[2] := "SubstanceID [uid]";
lvals[3] := "SourceName [src]";
lvals[4] := "CurrentSourceName [csn]";
lvals[5] := "SourceID [srid]";
lvals[6] := "SourceCategory [srcc]";
lvals[7] := "CompleteSynonym [csyn]";
lvals[8] := "Synonym [syno]";
lvals[9] := "Comment [cmt]";
lvals[10] := "TotalAidCount [tac]";
lvals[11] := "StandardizedCID [scid]";
lvals[12] := "ComponentCID [ccid]";
lvals[13] := "CompoundID [cid]";
lvals[14] := "AssaySourceName [asrc]";
lvals[15] := "StructureID [stid]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed') then
resize(lvals, 36);
lvals[1] := "All Fields [all]";
lvals[2] := "Title [titl]";
lvals[3] := "Text Word [word]";
lvals[4] := "MeSH Terms [mesh]";
lvals[5] := "MeSH Major Topic [majr]";
lvals[6] := "Author [auth]";
lvals[7] := "Journal [jour]";
lvals[8] := "Affiliation [affl]";
lvals[9] := "EC/RN Number [ecno]";
lvals[10] := "Supplementary Concept [subs]";
lvals[11] := "Volume [vol]";
lvals[12] := "Pagination [page]";
lvals[13] := "Publication Type [ptyp]";
lvals[14] := "Language [lang]";
lvals[15] := "Issue [iss]";
lvals[16] := "MeSH Subheading [subh]";
lvals[17] := "Secondary Source ID [si]";
lvals[18] := "Title/Abstract [tiab]";
lvals[19] := "Other Term [otrm]";
lvals[20] := "Investigator [invr]";
lvals[21] := "Author - Corporate [coln]";
lvals[22] := "Pharmacological Action [papx]";
lvals[23] := "Grant Number [grnt]";
lvals[24] := "Author - First [faut]";
lvals[25] := "Author - Full [full]";
lvals[26] := "Investigator - Full [finv]";
lvals[27] := "Transliterated Title [tt]";
lvals[28] := "Author - Last [laut]";
lvals[29] := "Location ID [lid]";
lvals[30] := "Book [book]";
lvals[31] := "Editor [ed]";
lvals[32] := "ISBN [isbn]";
lvals[33] := "Publisher [pubn]";
lvals[34] := "Author - Identifier [auid]";
lvals[35] := "Subject - Personal Name [ps]";
lvals[36] := "Conflict of Interest Statements [cois]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed Central') then
resize(lvals, 29);
lvals[1] := "All Fields [all]";
lvals[2] := "Author [auth]";
lvals[3] := "Title [titl]";
lvals[4] := "Abstract [abst]";
lvals[5] := "Figure/Table Caption [capt]";
lvals[6] := "Section Title [sect]";
lvals[7] := "Reference [refr]";
lvals[8] := "Reference Author [refa]";
lvals[9] := "Journal [jour]";
lvals[10] := "Volume [vol]";
lvals[11] := "Issue [iss]";
lvals[12] := "Pagination [page]";
lvals[13] := "Text Word [word]";
lvals[14] := "Body - All Words [arti]";
lvals[15] := "Body - Key Terms [kwd]";
lvals[16] := "Methods - Key Terms [meth]";
lvals[17] := "MeSH Terms [mesh]";
lvals[18] := "MeSH Major Topic [majr]";
lvals[19] := "MeSH Subheading [subh]";
lvals[20] := "EC/RN Number [ecno]";
lvals[21] := "Supplementary Concept [subs]";
lvals[22] := "Affiliation [affl]";
lvals[23] := "Organism [orgn]";
lvals[24] := "Organism unsynonymized [onsn]";
lvals[25] := "Accession [accn]";
lvals[26] := "DOI [doi]";
lvals[27] := "Full Author Name [full]";
lvals[28] := "Grant Number [grnt]";
lvals[29] := "Acknowledgments [ack]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed Health') then
resize(lvals, 18);
lvals[1] := "All Fields [all]";
lvals[2] := "Title [titl]";
lvals[3] := "Keyphrase [kyph]";
lvals[4] := "Secondary Text [stxt]";
lvals[5] := "Concept Phrases [conp]";
lvals[6] := "Book [book]";
lvals[7] := "ParentId [pid]";
lvals[8] := "PmId [pmid]";
lvals[9] := "Subject [sub]";
lvals[10] := "AccessionID [aid]";
lvals[11] := "UMLSID [umls]";
lvals[12] := "ICD9ID [icd9]";
lvals[13] := "BioconceptsID [bcid]";
lvals[14] := "DrugName [dr]";
lvals[15] := "Type [type]";
lvals[16] := "Category [cat]";
lvals[17] := "Publisher [publ]";
lvals[18] := "CollectionId [clid]";
SetLegalValues(lvals);
end if;
if (#database eq 'SNP') then
resize(lvals, 35);
lvals[1] := "All Fields [all]";
lvals[2] := "Reference SNP ID [rs]";
lvals[3] := "Chromosome [chr]";
lvals[4] := "Gene Name [gene]";
lvals[5] := "Submitter Handle [han]";
lvals[6] := "Accession [accn]";
lvals[7] := "LocusLink ID [llid]";
lvals[8] := "Organism [orgn]";
lvals[9] := "Function Class [fxn]";
lvals[10] := "Genotype [gtyp]";
lvals[11] := "non reference assembly [nref]";
lvals[12] := "Heterozygosity [hetz]";
lvals[13] := "Map Weight [mpwt]";
lvals[14] := "Validation Status [vali]";
lvals[15] := "Success Rate [srat]";
lvals[16] := "Create Build ID [cbid]";
lvals[17] := "Update Build ID [ubid]";
lvals[18] := "Population Class [pcls]";
lvals[19] := "Method Class [mcls]";
lvals[20] := "Submitter SNP ID [ss]";
lvals[21] := "Local SNP ID [sid]";
lvals[22] := "Allele [vari]";
lvals[23] := "SNP Class [scls]";
lvals[24] := "Gene Description [gdsc]";
lvals[25] := "Base Position [cpos]";
lvals[26] := "Contig Position [gpos]";
lvals[27] := "Text Word [word]";
lvals[28] := "Reference Amino Acid [wtaa]";
lvals[29] := "Variant Amino Acid [mtaa]";
lvals[30] := "Reference SNP [rsnp]";
lvals[31] := "SNP Index [sidx]";
lvals[32] := "SNP Allele Origin [alor]";
lvals[33] := "Suspected false variation [susp]";
lvals[34] := "Clinical Significance [clin]";
lvals[35] := "Global Minor Allele Frequency [gmaf]";
SetLegalValues(lvals);
end if;
if (#database eq 'SRA') then
resize(lvals, 18);
lvals[1] := "All Fields [all]";
lvals[2] := "Accession [accn]";
lvals[3] := "Title [titl]";
lvals[4] := "Properties [prop]";
lvals[5] := "Text Word [word]";
lvals[6] := "Organism [orgn]";
lvals[7] := "Author [auth]";
lvals[8] := "BioProject [gprj]";
lvals[9] := "BioSample [bspl]";
lvals[10] := "Platform [plat]";
lvals[11] := "Strategy [stra]";
lvals[12] := "Source [src]";
lvals[13] := "Selection [sel]";
lvals[14] := "Layout [lay]";
lvals[15] := "ReadLength [rlen]";
lvals[16] := "Access [acs]";
lvals[17] := "Aligned [aln]";
lvals[18] := "Mbases [mbs]";
SetLegalValues(lvals);
end if;
if (#database eq 'Sparcle') then
resize(lvals, 17);
lvals[1] := "All Fields [all]";
lvals[2] := "Status [stat]";
lvals[3] := "ECNumber [ecn]";
lvals[4] := "CDDShortname [shrt]";
lvals[5] := "GeneSymbol [gene]";
lvals[6] := "Name [nm]";
lvals[7] := "Label [lb]";
lvals[8] := "ECTitle [ectl]";
lvals[9] := "CDDTitle [cdtl]";
lvals[10] := "PDBTitle [pdtl]";
lvals[11] := "CDDDescription [cdds]";
lvals[12] := "BiosystemsDescription [bsds]";
lvals[13] := "GeneDescription [geds]";
lvals[14] := "Organism [orgn]";
lvals[15] := "Comment [cmmt]";
lvals[16] := "Defline [dfln]";
lvals[17] := "ReviewLevel [lev]";
SetLegalValues(lvals);
end if;
if (#database eq 'Structure') then
resize(lvals, 48);
lvals[1] := "All Fields [all]";
lvals[2] := "PDB Accession [accn]";
lvals[3] := "EC/RN Number [ecno]";
lvals[4] := "Resolution [reso]";
lvals[5] := "Experimental Method [expm]";
lvals[6] := "Title [titl]";
lvals[7] := "Abstract [abs]";
lvals[8] := "Author [auth]";
lvals[9] := "PDB Class [pcla]";
lvals[10] := "PDB Source [psrc]";
lvals[11] := "PDB Description [pdsc]";
lvals[12] := "PDB Comment [pcom]";
lvals[13] := "PDB Chemical Code [lcod]";
lvals[14] := "Chemical Name [lnam]";
lvals[15] := "Chemical Synonyms [csyn]";
lvals[16] := "Chemical Description [ldes]";
lvals[17] := "Organism [orgn]";
lvals[18] := "Taxonomy ID [txid]";
lvals[19] := "BioUnit Protein Molecule Count [pmc]";
lvals[20] := "BioUnit DNA Molecule Count [dmc]";
lvals[21] := "BioUnit RNA Molecule Count [rmc]";
lvals[22] := "BioUnit Biopolymer Count [bpc]";
lvals[23] := "BioUnit Chemical Count [lcou]";
lvals[24] := "BioUnit Other Molecule Count [ocou]";
lvals[25] := "Journal [jour]";
lvals[26] := "Conserved Domain PSSMID [cdid]";
lvals[27] := "Conserved Domain Short Name [cdsn]";
lvals[28] := "Conserved Domain Title [cddt]";
lvals[29] := "Conserved Domain Description [cddf]";
lvals[30] := "Conserved Domain Superfamily PSSMID [sfid]";
lvals[31] := "Conserved Domain Superfamily Short Name [spfn]";
lvals[32] := "Conserved Domain Superfamily Title [sptl]";
lvals[33] := "Conserved Domain Superfamily Description [spdf]";
lvals[34] := "Oligomeric State [os]";
lvals[35] := "Protein Name [pnam]";
lvals[36] := "Gene Name [gn]";
lvals[37] := "Gene Description [gdsc]";
lvals[38] := "DNA Name [dnam]";
lvals[39] := "RNA Name [rnam]";
lvals[40] := "Other Molecule Name [onam]";
lvals[41] := "ASU Protein Molecule Count [apmc]";
lvals[42] := "ASU DNA Molecule Count [admc]";
lvals[43] := "ASU RNA Molecule Count [armc]";
lvals[44] := "ASU Biopolymer Count [abpc]";
lvals[45] := "ASU Chemical Count [alct]";
lvals[46] := "ASU Other Molecule Count [aoct]";
lvals[47] := "BioUnit Molecular Weight [mlwt]";
lvals[48] := "Number of PDB Records per Structure [fc]";
SetLegalValues(lvals);
end if;
if (#database eq 'Taxonomy') then
resize(lvals, 16);
lvals[1] := "All Fields [all]";
lvals[2] := "Scientific Name [scin]";
lvals[3] := "Common Name [comn]";
lvals[4] := "Synonym [txsy]";
lvals[5] := "All Names [alln]";
lvals[6] := "Next Level [nxlv]";
lvals[7] := "Subtree [sbtr]";
lvals[8] := "Lineage [lnge]";
lvals[9] := "GC [gc]";
lvals[10] := "MGC [mgc]";
lvals[11] := "PGC [pgc]";
lvals[12] := "Division [txdv]";
lvals[13] := "Rank [rank]";
lvals[14] := "Properties [prop]";
lvals[15] := "Text Word [word]";
lvals[16] := "Name Tokens [ntok]";
SetLegalValues(lvals);
end if;
if (#database eq 'UniGene') then
resize(lvals, 25);
lvals[1] := "All Fields [all]";
lvals[2] := "Organism [orgn]";
lvals[3] := "Title [titl]";
lvals[4] := "Library [libr]";
lvals[5] := "Tissue [tiss]";
lvals[6] := "Clone ID [clon]";
lvals[7] := "Nucleotide Accession [ncac]";
lvals[8] := "Nucleotide UID [nuid]";
lvals[9] := "Protein Accession [prac]";
lvals[10] := "Protein UID [puid]";
lvals[11] := "Properties [prop]";
lvals[12] := "Text Word [word]";
lvals[13] := "Chromosome [chr]";
lvals[14] := "Gene Name [gene]";
lvals[15] := "Gene Description [gdsc]";
lvals[16] := "Gene ID [llid]";
lvals[17] := "Taxonomy ID [txid]";
lvals[18] := "Est Count [estc]";
lvals[19] := "mRNA Count [mrna]";
lvals[20] := "Sequence Count [seqc]";
lvals[21] := "Cluster ID [cid]";
lvals[22] := "Expression [expr]";
lvals[23] := "Restricted Expression [rexp]";
lvals[24] := "Page Rank [prnk]";
lvals[25] := "Record Type [rtyp]";
SetLegalValues(lvals);
end if;
if (#database eq 'dbGaP') then
resize(lvals, 40);
lvals[1] := "All Fields [all]";
lvals[2] := "Discriminator [disc]";
lvals[3] := "Object Type [obj]";
lvals[4] := "Ancestor [ance]";
lvals[5] := "Belongs To [belo]";
lvals[6] := "Attribution [attr]";
lvals[7] := "Is Root Study [rtst]";
lvals[8] := "Is Top-Level Study [tlst]";
lvals[9] := "Study ID [stid]";
lvals[10] := "Study Name [stnm]";
lvals[11] := "Disease [dis]";
lvals[12] := "Project [proj]";
lvals[13] := "Genotype Platform [geno]";
lvals[14] := "Study Has SRA Components [sra]";
lvals[15] := "Study [stud]";
lvals[16] := "Has Variable [hasv]";
lvals[17] := "Variable ID [vrid]";
lvals[18] := "Variable Name [vrnm]";
lvals[19] := "Variable Description [vrds]";
lvals[20] := "Variable [var]";
lvals[21] := "Has Document [hasd]";
lvals[22] := "Document ID [dcid]";
lvals[23] := "Document Name [dcnm]";
lvals[24] := "Document [doc]";
lvals[25] := "Document Part [docp]";
lvals[26] := "Has Analysis [hasa]";
lvals[27] := "Analysis ID [anid]";
lvals[28] := "Analysis Name [annm]";
lvals[29] := "Analysis [anls]";
lvals[30] := "Has Dataset [hast]";
lvals[31] := "Dataset ID [dsid]";
lvals[32] := "Dataset Name [dsnm]";
lvals[33] := "Dataset [ds]";
lvals[34] := "PhenX [px]";
lvals[35] := "Has PhenX Mapping [hasp]";
lvals[36] := "Study Archive [arch]";
lvals[37] := "Molecular Data Type [mole]";
lvals[38] := "Primary Phenotype in Study [prph]";
lvals[39] := "Data Access Committee [dac]";
lvals[40] := "Study Design [design]";
SetLegalValues(lvals);
end if;
if (#database eq 'BLAST Database') then
resize(lvals, 12);
lvals[1] := "All Fields [all]";
lvals[2] := "Database Name [db]";
lvals[3] := "Database Title [titl]";
lvals[4] := "Database Organism Taxid [orgn]";
lvals[5] := "Genome Collection Assembly Name [asm]";
lvals[6] := "Blast Sequence Type [seqt]";
lvals[7] := "Blast Sequence Strategy [seqs]";
lvals[8] := "Blast Database Source [src]";
lvals[9] := "Keyword [keyw]";
lvals[10] := "NCBI Genome Project ID [prj]";
lvals[11] := "Gpipe Build Name [gpb]";
lvals[12] := "NCBI WGS Project ID [wgpr]";
SetLegalValues(lvals);
end if;
Accession [accn]
Limits search by dates.
Date Range
Date Field
Selet a field to apply <i>Date Range</i>. The legal values are dependent on <i>Database</i>.
resize(lvals, 1);
lvals[1] := "";
SetLegalValues(lvals);
#database := Parameter('Database');
if (#database eq 'Assembly') then
resize(lvals, 5);
lvals[1] := "";
lvals[2] := "Date - Sequences Release [srdt]";
lvals[3] := "Date - Assembly Update [updt]";
lvals[4] := "Date - GenBank Assembly Release [grls]";
lvals[5] := "Date - RefSeq Assembly Release [rrls]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioProject') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Registration Date [date]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioSample') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'BioSystems') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "CreateDate [pdat]";
lvals[3] := "ModifyDate [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Books') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Year [pdat]";
lvals[3] := "Release Date [rd]";
SetLegalValues(lvals);
end if;
if (#database eq 'ClinVar') then
resize(lvals, 4);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
lvals[3] := "Creation Date [cdat]";
lvals[4] := "Last interpreted [imod]";
SetLegalValues(lvals);
end if;
if (#database eq 'Conserved Domains') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Core Nucleotide') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'EST') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'GEO DataSets') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Update Date [udat]";
SetLegalValues(lvals);
end if;
if (#database eq 'GSS') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'GTR') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Gene') then
resize(lvals, 4);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
lvals[3] := "Creation Date [cdat]";
lvals[4] := "Date Discontinued [ddat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Genome') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Create Date [cdt]";
SetLegalValues(lvals);
end if;
if (#database eq 'MedGen') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'NCBI Web Site') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'NLM Catalog') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Entrez Date [edat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Nucleotide') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'OMIM') then
resize(lvals, 5);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
lvals[3] := "Modification History [mdhs]";
lvals[4] := "Publication Date [pdat]";
lvals[5] := "Clinical Synopsis Date [csdt]";
SetLegalValues(lvals);
end if;
if (#database eq 'PopSet') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Probe') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Modification Date [mdat]";
lvals[3] := "Publication Date [pdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Protein') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Protein Clusters') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Creation Date [cdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem BioAssay') then
resize(lvals, 4);
lvals[1] := "";
lvals[2] := "Deposit Date [ddat]";
lvals[3] := "Modify Date [mdat]";
lvals[4] := "Hold Until Date [hdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem Compound') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "CreateDate [cdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubChem Substance') then
resize(lvals, 6);
lvals[1] := "";
lvals[2] := "SourceReleaseDate [srd]";
lvals[3] := "DepositDate [ddat]";
lvals[4] := "ModifyDate [mdat]";
lvals[5] := "HoldUntilDate [hud]";
lvals[6] := "AvailableDate [adat]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed') then
resize(lvals, 7);
lvals[1] := "";
lvals[2] := "Date - Publication [pdat]";
lvals[3] := "Date - Entrez [edat]";
lvals[4] := "Date - MeSH [mhda]";
lvals[5] := "Date - Modification [mdat]";
lvals[6] := "Date - Completion [cdat]";
lvals[7] := "Date - Create [crdt]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed Central') then
resize(lvals, 6);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Electronic Publication Date [epdt]";
lvals[4] := "PMC Live Date [ldat]";
lvals[5] := "Entrez Date [edat]";
lvals[6] := "Print Publication Date [ppdt]";
SetLegalValues(lvals);
end if;
if (#database eq 'PubMed Health') then
resize(lvals, 4);
lvals[1] := "";
lvals[2] := "ReleaseDate [rd]";
lvals[3] := "Date of publication [pdat]";
lvals[4] := "Update Date [udat]";
SetLegalValues(lvals);
end if;
if (#database eq 'SNP') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'SRA') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Publication Date [pdat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Sparcle') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "CreateDate [pdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Structure') then
resize(lvals, 4);
lvals[1] := "";
lvals[2] := "PDB Deposit Date [pdd]";
lvals[3] := "MMDB Entry Date [ddat]";
lvals[4] := "MMDB Modify Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'Taxonomy') then
resize(lvals, 3);
lvals[1] := "";
lvals[2] := "Entrez Date [edat]";
lvals[3] := "Modification Date [mdat]";
SetLegalValues(lvals);
end if;
if (#database eq 'BLAST Database') then
resize(lvals, 2);
lvals[1] := "";
lvals[2] := "Last Update [date]";
SetLegalValues(lvals);
end if;
Limits search on records bounded by two specific dates. For example, <b>2008/01/01-2008/06/30</b> or <b>2001-2003</b>. Or leave an empty value for no date limits.
SetDisabledDetails('To enable '.ParameterName().', select a Date Field first');
StrLength(Parameter('Date Field')) > 0;
<p>Limits a range of matching records to return. For example, <b>1-100</b> or <b>100-150</b>. The total number of matching records is provided in <i>matchingCount</i></p>
1
Specifies what to do with results which produced warnings.
Data to Fail Port
Data to Pass Port
Don't Pass Data
GroupType
NCBI Account
UsingParent
Use your NCBI account for high-frequency E-utilities requests.
NCBI Account API Key
NCBI Account Tool
NCBI Account Email
NCBI Account API Key
The API key of your NCBI account.
NCBI Account Tool
Specifies the software that is producing the request.
NCBI Account Email
The email of your NCBI account.
<p>
The word or phrase to search for, for example, <b>stem cells</b>.
</p>
<p>
<i>Text Query</i> accepts a combination of terms, fields, boolean operators to conduct advanced search. For example, use <b>"pancreatic cancer"[titl] AND JOP[jour]</b> and set <i>Database</i> to <b>PubMed</b> to search for pancreatic cancer in the title field of JOP (Journal of the Pancreas) articles. The <i>Search Field</i> is ignored for such advanced search. A more complex example is <b>transporter[title] AND ("Drosophila melanogaster"[orgn] OR "Mus musculus"[orgn])</b> for <b>Gene</b> database. Refer to <i>Search Field</i> to know the short fields names for each databse, that is the names inside [].
</p>
<p>
See <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/entrez/query/static/esearch_help.html')">NCBI ESearch utility help</a> for more details.
</p>
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
inputCache,searchResults
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentReturnsFail
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Entrez ESearch for Each Data
Protocol AutoLayout
1
0
DATABASENODE
Component Revision
100
The component this component is based upon.
SubProtocol
{CDED4568-2DF4-4B8D-9538-934A267E75A2}
90 -95
By Ref
1
100
Searches a Entrez database to retrieve primary IDs
<p>
Uses the Entrez ESearch utility. Select a Entrez <i>Database</i> to search for the text query specified in <i>Entrez Query Property</i> (usually named EntrezQuery). The search returns a list of primary IDs of the matching records in the database. Each of these IDs is sent out the Pass port in <i>primaryID</i>. The total number of matching records is provided in <i>matchingCount</i>.
</p>
<p>
Use <i>Search Field</i> and <i>Date Limits</i> to limit searching space for more refined results.
</p>
<p>
If there were no search results produced for your query, the input data record is sent out the Fail Port unchanged.
<br><br>
<p>
<b>Note regarding Entrez searches:</b>
<p>
Entrez is smart about searches so if your query does not succeed exactly as stated, it is sometimes reinterpreted more broadly in order to produce related results. If Entrez does reinterpret your query, it will issue a warning and the results data record will be tagged with a <b>warnings</b> property which will be set to <b>true</b>. There are also additional properties which will identify the problematic query terms.
<p>
You have three options for how this component will handle results with warnings, chosen with the <b>Results With Warnings</b> parameter:
<ul>
<li>Send to Pass Port - all results will come out the Pass Port.
<li>Send to Fail Port - results without warnings (exact matches to your query) will come out the Pass Port; results with warnings (matches to the broadened query) will come out the Fail Port.
<li>Don't Pass Data - no results with warnings will be produced.
</ul>
<br>
<p>
The output data record can be streamed into <a href="javascript:DoLink('{A116CE06-CF9D-4858-A746-C34598A24C10}')">Entrez ESummary</a>, <a href="javascript:DoLink('{9579C91D-05FA-4C2F-9D84-125CCB6B9542}')">Entrez EFetch</a>, and <a href="javascript:DoLink('{3114F126-CF80-4BD4-9D41-06BC91EA9A4E}')">Entrez ELink</a> to retrieve more information. Different database uses different primary IDs, so an ID must be used in conjunction with its database.
<p>
<b>See Also:</b> <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/entrez/query/static/esearch_help.html')">NCBI ESearch utility help</a>
<p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
20.1.0.2004
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Mon Jun 17 09:53:43 2019
Local Component Identifier
7
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
The original path within the database of this component.
Components\Biology\Sequence Analysis\Database Query\Entrez\Entrez ESearch for Each Data
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@warning := "Failed to retrieve GI for accession " . accessionNumber;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
390 -95
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
12
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
<p>Short text send to the DS client.</p>
<strong>Note:</strong>
<ul>
<li>Text is limited to 32 characters.</li>
</ul>
Client Message Final
<p>Text send to the DS Client.</p>
<strong>Note:</strong>
<ul>
<li>Text length is not limited.</li>
</ul>
Parameters controlling the DS report.
Report Error
Remove Duplicates from Summary
Report Detail
Report Warning
Report Summary
Text added to <strong>Summary</strong> section in the DS report.
Text added to <strong>Detail</strong> section in the DS report.
Text added to <strong>Warning</strong> section in the DS report.
$(Warning)
Text added to <strong>Error</strong> section in the DS report.
<strong>Note:</strong>
<ul>
<li>A specified text will change the run status to failure.</li>
</ul>
When set to True, all duplicate <strong>Summary</strong> lines in the DS report are removed.
True
False
Contains less commonly used parameters.
Debug Level
Debug Message
Message to send to the log file (if log is specified)
User can set the debug level for the debug message.
1
2
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The date this component was registered.
Mon Feb 26 16:26:52 2018
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Messaging
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{8F288684-93BC-4712-BC9A-73930339BDEB}
690 -95
By Ref
1
100
Sends text to the client and/or to the DS report
<p>Messages can be sent to the client or text can be specified for the DS report.</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
18.1.0.1604
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
Local Component Identifier
14
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\DS Messaging
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
debugMessage("Failed to retrieve sequence ");
for #i in 1 .. NumProperties() loop
debugMessage(IthPropertyName(#i) . " : " . IthProperty(#i));
end loop;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1365 -455
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
17
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
#status := true;
if(@NumSeqs = 0)then
#status := False;
ErrorMsg := @NumIds . " hits were found but no sequences could be retrieved from the NCBI server";
elsif (@NumSeqs < @NumIDs)then
WarningMsg := @NumIds . " hits were found but only " . @NumSeqs . " sequences could be retrieved from the NCBI server";
end if;
#status;
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -635
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
18
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
The property containing the location of the similarity search hits in the hierarchy.
SimilaritySearchHitsPackLocation
The property containing the pack ID for the similarity search hits. Used in reassembly.
SimilaritySearchHitsPackId
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The location of the similarity search hits in the hierarchy. The similarity search hits will be unpacked, and
passed out the Pass port, annotated with properties SimilaritySearchHitsPackId and SimilaritySearchHitsPackLocation.
/SciTegic.*/SciTegic.SearchResultGroup/*[contains(local-name(),'Group')]/
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Fri Dec 11 10:01:30 2009
Extract Similarity Search Hits
GeneratorNode
The component this component is based upon.
Evaluate Expression
{8FE8B2B4-92D1-4FB4-AB4A-762C80FDD304}
1290 -815
0
100
Extracts Similarity Search Hits from a Similarity Search Result record.
Generates a Sequence data record for each similarity search database hit from a similarity Search Result record for a particular sequence.
If you wish to reassemble the similarity Search Result record later, you should branch the incoming pipeline and apply a <I>Delete Hits from Similarity Search Result</I> component in parallel to obtain the parent; the parent can be combined later with the children using the <I>Reassemble Similarity Search Result</I>.
<strong>See Also:</strong> <a href="javascript:DoLink('sciapps/scitegic/seqanal/docs/seqanal.pdf')">Sequence Analysis User Guide</a>
Generic Data
Similarity Search Result records
Generic Data
Similarity Search Hits
Generic Data
If you turn this on, this port will contain records that are not Similarity Search Results.
The server version when this component was registered
4.5.2.100
Local Component Identifier
5
The original path within the database of this component.
Components\Biology\Sequence Analysis\Search and Similarity\Manipulators\Extract Similarity Search Hits
Short message send to the client.
Client Message Final
Final Message reported to the DS Client
Options controlling the DS report page
Report Error
Report Summary
Remove Duplicates from Summary
Report Warning
Report Detail
Message added to <strong>Summary</strong> section in DS report
Message added to <strong>Detail</strong> section in DS report
Message added to <strong>Warning</strong> section in DS report
Al1HAAE9TWVzc2FnZSBhZGRlZCB0byA8c3Ryb25nPkVycm9yPC9zdHJvbmc+IHNlY3Rpb24gaW4gRFMgcmVwb3J0LFN0cmluZ1R5cGUACAAAAAEBAAEABAD8AAAAAAAAAAAANFJlcG9ydCBFcnJvcgPMU8+7AS1GYWlsIHRvIGZldGNoIGhpdCBzZXF1ZW5jZXMgZnJvbSBOQ0JJIHNlcnZlcgA=
StringType
StringType
Message added to <strong>Error</strong> section in DS report
ErrorMsg
When set to True, all duplicate <strong>Summary</strong> lines in the DS report are removed.
True
False
Contains less commonly used parameters.
Debug Level
Debug Message
Message to send to the log file (if log is specified)
User can set the debug level for the debug message.
1
2
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The date this component was registered.
Mon Feb 26 16:26:52 2018
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Messaging
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{8F288684-93BC-4712-BC9A-73930339BDEB}
1890 -815
By Ref
1
100
Sends text to the client and/or to the DS report
<p>Messages can be sent to the client or text can be specified for the DS report.</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
18.1.0.1604
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
Local Component Identifier
19
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\DS Messaging
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
PDBCode, PDBChain
When the network runs, processes once or until done for each process call
True
False
Allow the subprotocol to execute batched and in parallel on one or more remote Pipeline Pilot servers. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
Batch Size
Server
Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
Comma separated list of Pipeline Pilot servers on which to execute the remote subprotocol. the server names should be in the same format as the Pipeline Pilot Client's change server dialog. However, specifying <b>'localhost'</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU systems and Pipeline Pilot Linux clusters.
Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Optional comma separated list of maximum number of batches that can simultaneously execute on each remote server. The number of entries in this parameter must equal the number of servers.
#processes := parameter();
expand(',', #processes);
if (numvalues(#processes) > 0) then
#servers := parameter('Server');
expand(',', #servers);
end if;
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ')');
numvalues(#processes) == 0 or numvalues(#processes) == numvalues(#servers);
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
SCOP and Ligands and Resolution
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
1590 -995
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
Use a subprotocol to collapse one or more components into a single item. Open a subprotocol to view or edit its contents.
To define input and output ports, right-click and select Edit. You can save subprotocols and reuse them. For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols Overview</a>.
Tip: Any component can be made into a shortcut, so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating Shortcuts</a>.
Generic
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
6.1.0.10
%VERSION 3%
2245 -1445 576 352
255 255 0
For PDB related database, creates HspSCOPs.txt and HspLigands.txt to store SCOP and ligand information for each BLAST hit. Also creats PDBResolution.txt to store PDB structure resolution.
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/core
The date this component was registered.
Thu Mar 08 14:43:51 2007
Local Component Identifier
9
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
PDBCode
Domain
SCOPFamilies
By PDB
By Sequence
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
Display Name
Find SCOP Families of PDB Domain
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
GUID
{A517D98A-3430-41DD-80AA-F311996827F6}
Component Point
938 -50
By Ref
1
Fancy Help
100
Finds SCOP families of a PDB domain
Use the property defined by <i>PDB Code Property</i> to specify a PDB code. Use the property defined by <i>Domain Property</i> to specify a domain in a format of <b>[chain]:[start residue number]-[end residue number]</b> or simply <b>[chain]:</b>. Use <i>Residue Sequence Number</i> to specify whether the residue numbers are from PDB residue sequence numbers or sequential numbers. A pre-compiled SCOP database is used for this searching. A new property defined by <i>SCOP Families Property</i> will be generated to output the names of the found SCOP families.
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
6.1.2.7
Component Sticky Notes
Protocol Sticky Notes
Runtime Identifier
9:0
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Tue Sep 04 10:30:44 2007
Subprotocol Made Link Into Copy
0
Location of the component in the database
Components\Accelrys Discovery Studio\Protein Modeling\Utilities\Find SCOP Families of PDB Domain
Local Component Identifier
0
Expression (or list of expressions separated by semi-colons) to evaluate
Initial Expression
Final Expression
Keep Calculated Properties
PDBSCOPs := PDBCode.':'.Domain.'=>'.SCOPFamilies;
This expression is executed only the first time data passes into the component. (Since there is no data available at initialization time, only expressions referencing global properties are allowed.)
This expression is executed only at finalization (after the last data passes out of the component).
(Since there is no data available at finalization time, only expressions referencing global properties are allowed.)
Adds any properties calculated in this component to a molecule's property list
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
Comment
Processes commands expressed in PilotScript syntax to apply rapid
manipulations to data record properties. In this way, the component may be
customized to build a new, specialized data manipulation component.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:05 2006
Component Usage
The 'Expression' is evaluated for each data record, and may alter and manipulate the properties of the data objects. The 'Initial expression' is evaluated the first time data passes into the component and is typically used for initializing global variables. See the online documentation for full details of PilotScript expression syntax.
Input - Data records
Output Pass - All Data records
Display Name
PDBSCOPs
The component this component is based upon.
Custom Manipulator (PilotScript)
Component RunTimeID
9:1
Component Icon
EXPRESSIONNODE
Disabled
0
GUID
{0995D675-7EE2-432C-95DF-E91C14052B22}
Component Point
1238 -50
Fancy Help
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Manipulators\Custom Manipulator (PilotScript)
Local Component Identifier
1
Comma-separated list of property names to keep
RenameList
PDBSCOPs
If provided, it causes the properties given in PropertyList to be
renamed to the corresponding name in this list. If not provided, then no properties are renamed.
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
the action to perform
Keep
The user who registered this component.
scitegic/core
The date this component was registered.
Tue Oct 17 16:03:31 2006
Component Comment
Removes all but the specified properties from the property list.
Component Usage
'PropertyList' contains a comma-separated list of the property names that you wish to keep. All other properties will be removed.
Input - Data records
Output Pass - All Data records
Display Name
Keep Properties
Protocol AutoLayout
true
Protocol ShowComment
false
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
EXPRESSIONNODE
Component Revision
100
The component this component is based upon.
SubProtocol
Protocol Scale
0
Component Point
1538 -50
GUID
{456195EF-9708-4367-B3DD-21EBF3796604}
Fancy Help
100
Keeps only the specified properties
For each incoming data record, retains only the properties found in the list of property names specified by <i>PropertyList</i>. All other properties are removed.
<p><b>Note:</b>
<ul>
<li>Use <i>RenameList</i> if you wish to change the names of the retained properties.
</ul>
<p><b>See Also:</b></p>
<ul>
<li>To remove specified properties from each data record, use
<a href="javascript:DoLink('{C2ED35FF-5397-4877-AADB-10F1A15D60E2}')">Remove Properties</a>.
</ul>
Generic
Generic
With only the specified properties remaining
None
The server version when this component was registered
6.0.1.4
Component Sticky Notes
By Ref
0
Protocol Sticky Notes
Runtime Identifier
9:2
Location of the component in the database
Components\Generic\Manipulators\Keep Properties
Local Component Identifier
2
The filename for the destination data. Be sure to specify an extension such as .txt or .xls. To save in compressed (zipped) format, add the extension .gz or .zip to the filename.
Maximum
IfFileExists
CharacterEncoding
$(RunDirectory)/HspSCOPs.txt
The maximum number of data records to write (all if value is empty)
What to do if the given destination file already exists
Overwrite
Append
Halt
<p>Indicates the character set that will be used within the file.</p>
<p>ANSI is the default and specifies a character set where each character is defined by a single byte. This allows the Latin-1 (ISO-8859-1) character set to be written to the file. Virtually all applications can read these files.</p>
<p>UTF-8 specifies a variable length character set. Each character is represented by 1 to 6 bytes. This allows writing out all extended characters specified in the Unicode character sets while maintaining reasonably lengthed files. Many applications do not support UTF-8 files.</p>
<p>"Unicode" and "Unicode big endian" are sets where each character is represented by exactly 2 bytes (also known as UCS-2). Depending on which is used will determine the order in which the bytes appear for each character. Many applications do not support Unicode or Unicode big endian files.</p>
<p>For more information about each of these ecoding styles, go to <a href="javascript:DoLink('http://www.unicode.org')">http://www.unicode.org</a></p>
ANSI
UTF-8
Unicode
Unicode big endian
Specifies the character to use to separate
the fields in the file. (Whole records are separated by line breaks.) Make sure you use a different
MultiValueDelimiter than Delimiter!
MultipleValueDelimiter
Tab
Space
Comma
Colon
Semicolon
The delimiter to use between multiple value on a single property. If "None",
multiple values are separated into their own columns.
None
Semicolon
Colon
Comma
Linefeed
OutputColumnNames
OutputColumnTypes
OutputRowNumber
UndefinedString
MissingValue Format
Text to output when data for a given column is not present (leave empty or enter text of choice)
Output column names
True
False
Output a row that includes column types
None
SpotfireFormat
NDIFormat
Includes the number of the record as the first field for each record
True
False
How to write out explicitly-defined MissingValue properties. Each missing value property
contains a string that may provide information about the reason the value is missing. You can ask to
see either the missing value text containing this string, or use the undefined string to write out missing values.
Use MissingValue Text
Use UndefinedString
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Comment
Writes data records to a text file, separating the records with a new line, and the individual fields within a record with the Delimiter character. The property names for the fields are stored as the first record. If a property has more than one value, it is written as multiple fields (for example, prop[0], prop[1] ...)
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 15:05:11 2006
Component Usage
Writes data records to a text file, separating the records with a new line and the individual fields within a record with the 'Delimiter' character. The property names for the fields will be stored as the first line.
'MultipleValueDelimiter' controls properties with more than one value. If None, then they are written as multiple fields (e.g., prop[1], prop[2] ...). Else, they are written into a single value separated by the chosen delimiter.
'UndefinedString' is the string to write out to represent undefined values.
Input - Data records
Advanced Parameters:
'OutputColumnNames' determines whether the first line of the file will be the column names.
'OutputColumnTypes' determines whether the next line of the file will be column type designations.
'OutputRowNumber' determines whether the first column of the file will be reserved for the Row number.
Display Name
Delimited Text Writer
The component this component is based upon.
Delimited Writer
Component RunTimeID
9:3
Disabled
0
Component Icon
DOCUMENTNODE
GUID
{9B0081A4-6085-4FF0-BD12-AEB8B7BD9CB5}
Component Point
1813 -50
Fancy Help
100
Writes data records to a text file with one data record per line and data separated by delimiters
<p>Data records are separated with a new line and the individual properties within each record are separated with the character specified by <i>Delimiter</I>. The property names for the fields are stored as the first line of the file.
<p>
<b>Note:</b>
<ul>
<li>Use <i>MultipleValueDelimiter</i> to control the output format for properties
with more than one value (array properties).
</ul>
Generic
None
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Writers\Delimited Text Writer
Local Component Identifier
3
The location of a node in the hierarchy; the children of this node will be unpacked, and
passed out the Pass port, annotated with properties PackId and Location. The remaining parts of the data record
will be passed out the Fail port, annotated with PackId.
/SciTegic.Sequence/SciTegic.HighScoringPairGroup/
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Whether to output the parent structure or the children
Parent
Children
The user who registered this component.
scitegic/core
The date this component was registered.
Fri Nov 19 09:32:47 2004
Component Usage
A new property (TagName) is created for each incoming data record, and it is defined with the value of true. Records tagged in this way can easily be identified in future components by evaluating whether the new property is defined. This is useful for marking a data stream for learning, substructure search, and other components.
Input - Data records
Output Pass - Data records with the tag defined.
Display Name
Unpack Children
Component Icon
ComponentNode
The component this component is based upon.
Evaluate Expression
GUID
{ACCDEEEF-1ED7-4B27-B263-BA531EABFC9C}
Component Point
363 -250
Disabled
0
Fancy Help
100
Unpacks the children beneath a specified location in the hierarchy into individual nodes
This component unpacks the children (if any) underneath the specified node and returns them through the Pass port. If the location is not found, the data record is passed unchanged through the Fail port.
The property <I>PackId</I> will be defined on the children. It will contain a number that is the sequence number of the original incoming data record. (If <i>PackId</i> is already defined on the incoming data record, then the current sequence number will be added as another element in the <i>PackId</i> array. This is useful in multiple unpack/repack steps.)
The property <I>Location</I> will be defined on the children, and will contain the location in the original record where it was detached from. (If <i>Location</i> is already defined, then the current location will be added as another element in the <i>Location</i> array. This is useful in multiple unpack/repack steps.)
These annotations give the information needed to repack the children back onto a copy of the parent (which can be gotten using a Delete Children component with the same location, in parallel).
See also <a href="javascript:DoLink('{ACCDEEEF-FEED-4B27-B263-BA531EABFC9C}')">Delete Children</a> and <a href="javascript:DoLink('{FEFEAAAA-1ED7-4B27-B263-BA531EABFC9C}')">Pack Children</a>.
Generic Data
Hierarchical data records
Generic Data
Data records containing the children nodes
Generic Data
The remaining data record after removing the children
The server version when this component was registered
4.5.2.0
Component Sticky Notes
Runtime Identifier
9:4
Location of the component in the database
Components\Generic\Utilities\Data Tree Manipulators\Unpack Children
Local Component Identifier
4
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
if hspHitStart defined then
#from := hspHitStart;
elsif property('hit-from') defined then
#from := property('hit-from');
end if;
if hspHitEnd defined then
#to := hspHitEnd;
elsif property('hit-to') defined then
#to := property('hit-to');
end if;
if (#from defined) and (#to defined) then
PDBCode := @PDBCode;
Domain := @PDBChain.':'.#from.'-'.#to;
True;
else
False;
end if;
An expression evaluated only when the component
is first inialized (when the first data record enters).
(Since there is no data available at initialization
time, only expressions referencing global
properties are allowed.)
An expression evaluated only when the
component is finalized (once the last data
record has passed).
*Since there is no data available at finalization
time, only expressions referencing global
properties are allowed.
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Version of expression parser to use.
2
Comment
Allows you to filter using the result from a PilotScript expression. The expression you specify is evaluated for each incoming data record, and is evaluated to a True or False value. If multiple statements are present, then the value of the last statement is used.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:18 2006
Component Usage
The 'Expression' is evaluated for each incoming data record. The last statement in the expression evaluates to a boolean value (true/false or nonzero/zero) and determines the fate of the data record: Records which evaluate to 'true' are passed out the Pass port; those which evaluate to 'false' are passed out the 'Fail' port.
Input - Data records
Output Pass - Data records which evaluate 'true'
Output Fail - Data records which evaluate 'false'
Display Name
Hit from and to
Component Icon
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
Disabled
0
GUID
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
Component Point
663 -250
Fancy Help
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Runtime Identifier
9:5
Location of the component in the database
Components\Generic\Filters\Custom Filter (PilotScript)
Local Component Identifier
5
6.0.2.0
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
/* gi|3402063|pdb|1A6V|L */
#hitid := RSPLIT(accessionNumber, '\|','g');
if(numvalues(#hitid) == 5) and (#hitid[3] eq 'PDB') then
@PDBCode := #hitid[4];
@PDBChain := #hitid[5];
if(@PDBChain eq ' ') then
@PDBChain := '';
end if;
True;
else
False;
end if;
An expression evaluated only when the component
is first inialized (when the first data record enters).
(Since there is no data available at initialization
time, only expressions referencing global
properties are allowed.)
An expression evaluated only when the
component is finalized (once the last data
record has passed).
*Since there is no data available at finalization
time, only expressions referencing global
properties are allowed.
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Version of expression parser to use.
2
Comment
Allows you to filter using the result from a PilotScript expression. The expression you specify is evaluated for each incoming data record, and is evaluated to a True or False value. If multiple statements are present, then the value of the last statement is used.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:18 2006
Component Usage
The 'Expression' is evaluated for each incoming data record. The last statement in the expression evaluates to a boolean value (true/false or nonzero/zero) and determines the fate of the data record: Records which evaluate to 'true' are passed out the Pass port; those which evaluate to 'false' are passed out the 'Fail' port.
Input - Data records
Output Pass - Data records which evaluate 'true'
Output Fail - Data records which evaluate 'false'
Display Name
set PDBCode and PDBChain
Component Icon
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
Disabled
0
GUID
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
Component Point
88 -425
Fancy Help
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Runtime Identifier
9:6
Location of the component in the database
Components\Generic\Filters\Custom Filter (PilotScript)
Local Component Identifier
6
6.0.2.0
PDBCode
Domain
Ligands
By PDB
By Sequence
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
Display Name
Find Ligands of PDB Domain
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
GUID
{45CB5E7D-4034-4BC3-AE25-4E82B9D85383}
Component Point
938 -425
By Ref
1
Fancy Help
100
Finds ligands within 5 Å of a PDB domain
Use the property defined by <i>PDB Code Property</i> to specify a PDB code. Use the property defined by <i>Domain Property</i> to specify a domain in a format of <b>[chain]:[start residue number]-[end residue number]</b> or simply <b>[chain]:</b>. Use <i>Residue Sequence Number</i> to specify whether the residue numbers are from PDB residue sequence numbers or sequential numbers. A ligand database pre-compiled from all PDB files is used for this searching. A new property defined by <i>Ligands Property</i> will be generated to output the names of the found ligands.
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
6.1.2.7
Component Sticky Notes
Protocol Sticky Notes
Runtime Identifier
9:7
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Tue Sep 04 10:30:40 2007
Subprotocol Made Link Into Copy
0
Location of the component in the database
Components\Accelrys Discovery Studio\Protein Modeling\Utilities\Find Ligands of PDB Domain
Local Component Identifier
7
Expression (or list of expressions separated by semi-colons) to evaluate
Initial Expression
Final Expression
Keep Calculated Properties
PDBLigands := PDBCode.':'.Domain.'=>'.Ligands;
This expression is executed only the first time data passes into the component. (Since there is no data available at initialization time, only expressions referencing global properties are allowed.)
This expression is executed only at finalization (after the last data passes out of the component).
(Since there is no data available at finalization time, only expressions referencing global properties are allowed.)
Adds any properties calculated in this component to a molecule's property list
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
Comment
Processes commands expressed in PilotScript syntax to apply rapid
manipulations to data record properties. In this way, the component may be
customized to build a new, specialized data manipulation component.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:05 2006
Component Usage
The 'Expression' is evaluated for each data record, and may alter and manipulate the properties of the data objects. The 'Initial expression' is evaluated the first time data passes into the component and is typically used for initializing global variables. See the online documentation for full details of PilotScript expression syntax.
Input - Data records
Output Pass - All Data records
Display Name
PDBLigands
The component this component is based upon.
Custom Manipulator (PilotScript)
Component RunTimeID
9:8
Component Icon
EXPRESSIONNODE
Disabled
0
GUID
{0995D675-7EE2-432C-95DF-E91C14052B22}
Component Point
1238 -425
Fancy Help
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Manipulators\Custom Manipulator (PilotScript)
Local Component Identifier
8
Comma-separated list of property names to keep
RenameList
PDBLigands
If provided, it causes the properties given in PropertyList to be
renamed to the corresponding name in this list. If not provided, then no properties are renamed.
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
the action to perform
Keep
The user who registered this component.
scitegic/core
The date this component was registered.
Tue Oct 17 16:03:31 2006
Component Comment
Removes all but the specified properties from the property list.
Component Usage
'PropertyList' contains a comma-separated list of the property names that you wish to keep. All other properties will be removed.
Input - Data records
Output Pass - All Data records
Display Name
Keep Properties
Protocol AutoLayout
true
Protocol ShowComment
false
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
EXPRESSIONNODE
Component Revision
100
The component this component is based upon.
SubProtocol
Protocol Scale
0
Component Point
1538 -425
GUID
{456195EF-9708-4367-B3DD-21EBF3796604}
Fancy Help
100
Keeps only the specified properties
For each incoming data record, retains only the properties found in the list of property names specified by <i>PropertyList</i>. All other properties are removed.
<p><b>Note:</b>
<ul>
<li>Use <i>RenameList</i> if you wish to change the names of the retained properties.
</ul>
<p><b>See Also:</b></p>
<ul>
<li>To remove specified properties from each data record, use
<a href="javascript:DoLink('{C2ED35FF-5397-4877-AADB-10F1A15D60E2}')">Remove Properties</a>.
</ul>
Generic
Generic
With only the specified properties remaining
None
The server version when this component was registered
6.0.1.4
Component Sticky Notes
By Ref
0
Protocol Sticky Notes
Runtime Identifier
9:9
Location of the component in the database
Components\Generic\Manipulators\Keep Properties
Local Component Identifier
9
The filename for the destination data. Be sure to specify an extension such as .txt or .xls. To save in compressed (zipped) format, add the extension .gz or .zip to the filename.
Maximum
IfFileExists
CharacterEncoding
$(RunDirectory)/HspLigands.txt
The maximum number of data records to write (all if value is empty)
What to do if the given destination file already exists
Overwrite
Append
Halt
<p>Indicates the character set that will be used within the file.</p>
<p>ANSI is the default and specifies a character set where each character is defined by a single byte. This allows the Latin-1 (ISO-8859-1) character set to be written to the file. Virtually all applications can read these files.</p>
<p>UTF-8 specifies a variable length character set. Each character is represented by 1 to 6 bytes. This allows writing out all extended characters specified in the Unicode character sets while maintaining reasonably lengthed files. Many applications do not support UTF-8 files.</p>
<p>"Unicode" and "Unicode big endian" are sets where each character is represented by exactly 2 bytes (also known as UCS-2). Depending on which is used will determine the order in which the bytes appear for each character. Many applications do not support Unicode or Unicode big endian files.</p>
<p>For more information about each of these ecoding styles, go to <a href="javascript:DoLink('http://www.unicode.org')">http://www.unicode.org</a></p>
ANSI
UTF-8
Unicode
Unicode big endian
Specifies the character to use to separate
the fields in the file. (Whole records are separated by line breaks.) Make sure you use a different
MultiValueDelimiter than Delimiter!
MultipleValueDelimiter
Tab
Space
Comma
Colon
Semicolon
The delimiter to use between multiple value on a single property. If "None",
multiple values are separated into their own columns.
None
Semicolon
Colon
Comma
Linefeed
OutputColumnNames
OutputColumnTypes
OutputRowNumber
UndefinedString
MissingValue Format
Text to output when data for a given column is not present (leave empty or enter text of choice)
Output column names
True
False
Output a row that includes column types
None
SpotfireFormat
NDIFormat
Includes the number of the record as the first field for each record
True
False
How to write out explicitly-defined MissingValue properties. Each missing value property
contains a string that may provide information about the reason the value is missing. You can ask to
see either the missing value text containing this string, or use the undefined string to write out missing values.
Use MissingValue Text
Use UndefinedString
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Comment
Writes data records to a text file, separating the records with a new line, and the individual fields within a record with the Delimiter character. The property names for the fields are stored as the first record. If a property has more than one value, it is written as multiple fields (for example, prop[0], prop[1] ...)
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 15:05:11 2006
Component Usage
Writes data records to a text file, separating the records with a new line and the individual fields within a record with the 'Delimiter' character. The property names for the fields will be stored as the first line.
'MultipleValueDelimiter' controls properties with more than one value. If None, then they are written as multiple fields (e.g., prop[1], prop[2] ...). Else, they are written into a single value separated by the chosen delimiter.
'UndefinedString' is the string to write out to represent undefined values.
Input - Data records
Advanced Parameters:
'OutputColumnNames' determines whether the first line of the file will be the column names.
'OutputColumnTypes' determines whether the next line of the file will be column type designations.
'OutputRowNumber' determines whether the first column of the file will be reserved for the Row number.
Display Name
Delimited Text Writer
The component this component is based upon.
Delimited Writer
Component RunTimeID
9:10
Disabled
0
Component Icon
DOCUMENTNODE
GUID
{9B0081A4-6085-4FF0-BD12-AEB8B7BD9CB5}
Component Point
1813 -425
Fancy Help
100
Writes data records to a text file with one data record per line and data separated by delimiters
<p>Data records are separated with a new line and the individual properties within each record are separated with the character specified by <i>Delimiter</I>. The property names for the fields are stored as the first line of the file.
<p>
<b>Note:</b>
<ul>
<li>Use <i>MultipleValueDelimiter</i> to control the output format for properties
with more than one value (array properties).
</ul>
Generic
None
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Writers\Delimited Text Writer
Local Component Identifier
10
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
PDBCode := @PDBCode;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
Comment
Processes commands expressed in PilotScript syntax to apply rapid
manipulations to data record properties. In this way, the component may be
customized to build a new, specialized data manipulation component.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Apr 23 12:20:45 2007
Component Usage
The 'Expression' is evaluated for each data record, and may alter and manipulate the properties of the data objects. The 'Initial expression' is evaluated the first time data passes into the component and is typically used for initializing global variables. See the online documentation for full details of PilotScript expression syntax.
Input - Data records
Output Pass - All Data records
Display Name
Custom Manipulator (PilotScript)
The component this component is based upon.
Custom Manipulator (PilotScript)
Component RunTimeID
9:11
Component Icon
EXPRESSIONNODE
Disabled
0
GUID
{0995D675-7EE2-432C-95DF-E91C14052B22}
Component Point
363 -775
Fancy Help
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
6.1.0.17
Component Sticky Notes
Location of the component in the database
Components\Generic\Manipulators\Custom Manipulator (PilotScript)
Local Component Identifier
11
PDBCode
Resolution
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p><b>Note:</b> This is a prototype feature, which we are making available
because we feel it is a useful technique to improve data throughput speeds in many situations. In particular, it can help to optimize resource use on multi-CPU and multi-core machines and on server clusters.
We welcome your feedback on scenarios where it works well, scenarios where it does not, and any other opinions you have about this utility.
</p><hr/><p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware)
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
Display Name
Find PDB Resolution
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
GUID
{F63A6BAA-E323-48FF-B5C3-532CF82DEA23}
Component Point
663 -775
By Ref
1
Fancy Help
100
Finds the resolution of a PDB Code
Use the property defined by <i>PDB Code Property</i> to specify a PDB code. A new property defined by <i>Resolution Property</i> will be generated to output the resolution of the PDB code.
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
6.1.2.7
Component Sticky Notes
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Fri Aug 31 18:40:51 2007
Runtime Identifier
9:12
Subprotocol Made Link Into Copy
0
Location of the component in the database
Components\Accelrys Discovery Studio\Protein Modeling\Utilities\Find PDB Resolution
Local Component Identifier
12
Expression (or list of expressions separated by semi-colons) to evaluate
Initial Expression
Final Expression
Keep Calculated Properties
PDBResolution := PDBCode.'=>'.Resolution;
This expression is executed only the first time data passes into the component. (Since there is no data available at initialization time, only expressions referencing global properties are allowed.)
This expression is executed only at finalization (after the last data passes out of the component).
(Since there is no data available at finalization time, only expressions referencing global properties are allowed.)
Adds any properties calculated in this component to a molecule's property list
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
Comment
Processes commands expressed in PilotScript syntax to apply rapid
manipulations to data record properties. In this way, the component may be
customized to build a new, specialized data manipulation component.
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 10:20:05 2006
Component Usage
The 'Expression' is evaluated for each data record, and may alter and manipulate the properties of the data objects. The 'Initial expression' is evaluated the first time data passes into the component and is typically used for initializing global variables. See the online documentation for full details of PilotScript expression syntax.
Input - Data records
Output Pass - All Data records
Display Name
Resolution
The component this component is based upon.
Custom Manipulator (PilotScript)
Component RunTimeID
9:13
Component Icon
EXPRESSIONNODE
Disabled
0
GUID
{0995D675-7EE2-432C-95DF-E91C14052B22}
Component Point
938 -775
Fancy Help
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax and can only be accessed by token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Manipulators\Custom Manipulator (PilotScript)
Local Component Identifier
13
Comma-separated list of property names to keep
RenameList
PDBResolution
If provided, it causes the properties given in PropertyList to be
renamed to the corresponding name in this list. If not provided, then no properties are renamed.
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
the action to perform
Keep
The user who registered this component.
scitegic/core
The date this component was registered.
Tue Oct 17 16:03:31 2006
Component Comment
Removes all but the specified properties from the property list.
Component Usage
'PropertyList' contains a comma-separated list of the property names that you wish to keep. All other properties will be removed.
Input - Data records
Output Pass - All Data records
Display Name
Keep Properties
Protocol AutoLayout
true
Protocol ShowComment
false
Add Protocol Comment Here
Add Protocol Comment Here
Disabled
0
Component Icon
EXPRESSIONNODE
Component Revision
100
The component this component is based upon.
SubProtocol
Protocol Scale
0
Component Point
1238 -775
GUID
{456195EF-9708-4367-B3DD-21EBF3796604}
Fancy Help
100
Keeps only the specified properties
For each incoming data record, retains only the properties found in the list of property names specified by <i>PropertyList</i>. All other properties are removed.
<p><b>Note:</b>
<ul>
<li>Use <i>RenameList</i> if you wish to change the names of the retained properties.
</ul>
<p><b>See Also:</b></p>
<ul>
<li>To remove specified properties from each data record, use
<a href="javascript:DoLink('{C2ED35FF-5397-4877-AADB-10F1A15D60E2}')">Remove Properties</a>.
</ul>
Generic
Generic
With only the specified properties remaining
None
The server version when this component was registered
6.0.1.4
Component Sticky Notes
By Ref
0
Protocol Sticky Notes
Runtime Identifier
9:14
Location of the component in the database
Components\Generic\Manipulators\Keep Properties
Local Component Identifier
14
The filename for the destination data. Be sure to specify an extension such as .txt or .xls. To save in compressed (zipped) format, add the extension .gz or .zip to the filename.
Maximum
IfFileExists
CharacterEncoding
$(RunDirectory)/PDBResolution.txt
The maximum number of data records to write (all if value is empty)
What to do if the given destination file already exists
Overwrite
Append
Halt
<p>Indicates the character set that will be used within the file.</p>
<p>ANSI is the default and specifies a character set where each character is defined by a single byte. This allows the Latin-1 (ISO-8859-1) character set to be written to the file. Virtually all applications can read these files.</p>
<p>UTF-8 specifies a variable length character set. Each character is represented by 1 to 6 bytes. This allows writing out all extended characters specified in the Unicode character sets while maintaining reasonably lengthed files. Many applications do not support UTF-8 files.</p>
<p>"Unicode" and "Unicode big endian" are sets where each character is represented by exactly 2 bytes (also known as UCS-2). Depending on which is used will determine the order in which the bytes appear for each character. Many applications do not support Unicode or Unicode big endian files.</p>
<p>For more information about each of these ecoding styles, go to <a href="javascript:DoLink('http://www.unicode.org')">http://www.unicode.org</a></p>
ANSI
UTF-8
Unicode
Unicode big endian
Specifies the character to use to separate
the fields in the file. (Whole records are separated by line breaks.) Make sure you use a different
MultiValueDelimiter than Delimiter!
MultipleValueDelimiter
Tab
Space
Comma
Colon
Semicolon
The delimiter to use between multiple value on a single property. If "None",
multiple values are separated into their own columns.
None
Semicolon
Colon
Comma
Linefeed
OutputColumnNames
OutputColumnTypes
OutputRowNumber
UndefinedString
MissingValue Format
Text to output when data for a given column is not present (leave empty or enter text of choice)
Output column names
True
False
Output a row that includes column types
None
SpotfireFormat
NDIFormat
Includes the number of the record as the first field for each record
True
False
How to write out explicitly-defined MissingValue properties. Each missing value property
contains a string that may provide information about the reason the value is missing. You can ask to
see either the missing value text containing this string, or use the undefined string to write out missing values.
Use MissingValue Text
Use UndefinedString
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Comment
Writes data records to a text file, separating the records with a new line, and the individual fields within a record with the Delimiter character. The property names for the fields are stored as the first record. If a property has more than one value, it is written as multiple fields (for example, prop[0], prop[1] ...)
The user who registered this component.
scitegic/core
The date this component was registered.
Mon Dec 11 15:05:11 2006
Component Usage
Writes data records to a text file, separating the records with a new line and the individual fields within a record with the 'Delimiter' character. The property names for the fields will be stored as the first line.
'MultipleValueDelimiter' controls properties with more than one value. If None, then they are written as multiple fields (e.g., prop[1], prop[2] ...). Else, they are written into a single value separated by the chosen delimiter.
'UndefinedString' is the string to write out to represent undefined values.
Input - Data records
Advanced Parameters:
'OutputColumnNames' determines whether the first line of the file will be the column names.
'OutputColumnTypes' determines whether the next line of the file will be column type designations.
'OutputRowNumber' determines whether the first column of the file will be reserved for the Row number.
Display Name
Delimited Text Writer
The component this component is based upon.
Delimited Writer
Component RunTimeID
9:15
Disabled
0
Component Icon
DOCUMENTNODE
GUID
{9B0081A4-6085-4FF0-BD12-AEB8B7BD9CB5}
Component Point
1538 -775
Fancy Help
100
Writes data records to a text file with one data record per line and data separated by delimiters
<p>Data records are separated with a new line and the individual properties within each record are separated with the character specified by <i>Delimiter</I>. The property names for the fields are stored as the first line of the file.
<p>
<b>Note:</b>
<ul>
<li>Use <i>MultipleValueDelimiter</i> to control the output format for properties
with more than one value (array properties).
</ul>
Generic
None
None
The server version when this component was registered
6.0.2.0
Component Sticky Notes
Location of the component in the database
Components\Generic\Writers\Delimited Text Writer
Local Component Identifier
15
<p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.</p><p>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
#rtcForEachCategory := (Parameter('RunToCompletion') eq 'True - For Each Category');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
elsif (#rtcForEachCategory) then
SetDisabledDetails("This parameter is not applicable when 'RunToCompletion' is set to 'True - For Each Category'");
end if;
#parallel and not #rtcForEachCategory;
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Generate Sequence Profile from Hits
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
1290 -1355
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them. For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut, so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating Shortcuts</a>.
</p>
Generic
None
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
7.9.0.30
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Tue Jan 26 10:02:05 2010
Local Component Identifier
30
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
The number of top hits to be included into the alignment. Use an empty value to include all hits.
If True, the gaps in query sequence and their corresponding residues or bases in hit sequences are removed out of the output alignment, which, then, can be used as a profile of the query sequence.
True
False
Temporary files converted by the network to true temporary filenames
tmpFasta
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
QuerySequence, QueryName, QueryDescription, JoinProperty, tmpCache
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Tue Oct 05 16:10:15 2010
{8254E352-764A-427E-B771-9908C4F7CD13}
Convert All Hits to Alignment
Protocol AutoLayout
true
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
90 -95
Protocol Scale
0
By Ref
1
100
Converts all BLAST hits to an alignment.
Reads a BLAST data record and convert all hits into an alignment data record with the BLAST query sequence put at the first sequence.
Generic
BLAST data record
Generic
Alignment data record
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
7.5.0.10
Protocol Sticky Notes
Protocol Options
Protocol Scale
100
Local Component Identifier
0
Components\Biology\Sequence Analysis\Search and Similarity\Manipulators\Convert All Hits to Alignment
The name of the aligmment file to generate
Keep Chain Breaks
If File Exists
Maximum
File Format
Keep Features
$(RunDirectory)/$(SequenceFileName)_profile.pir
The file format to use
Fasta
PIR
BSML
Whether to keep chain breaks or not
True
False
Keep sequence features in the output file generated.
SetDisabledDetails("This parameter is only applicable if 'File Format' is 'BSML'");
Parameter('File Format') eq 'BSML';
True
False
What to do if the given destination file already exists
Overwrite
Append
Halt
The maximum number of alignments to write (all if value is empty)
Temporary files converted by the network to true temporary filenames
tmpFasta, TmpGFF, tmpPir,tmpBSMLDir
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
originalFastaIDs, DisplayIDs
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/dscore
The date this component was registered.
Tue Aug 16 13:07:35 2011
{38EDF8BA-DA22-40FE-9F92-6F0F5767E9DA}
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Alignment Writer
Protocol AutoLayout
true
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
DOCUMENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
378 -95
Protocol Scale
0
By Ref
1
100
Writes a alignment file
Save input alignment records as entries in the flat file specified in <i>Destination</i>. The available formats include PIR, BSML and Fasta, as specified in <i>File Format</i>.
Generic
Alignment data record
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
8.5.0.200
Protocol Sticky Notes
Protocol Options
Make Shortcut by Default
Owner Access Only
Protocol Scale
100
Local Component Identifier
1
Subprotocol Made Link Into Copy
0
Components\Biology\Protein Modeling\Writers\DS Alignment Writer
Short message send to the client.
Client Message Final
Final Message reported to the DS Client
Options controlling the DS report page
Report Error
Report Summary
Remove Duplicates from Summary
Report Warning
Report Detail
Message added to <strong>Summary</strong> section in DS report
Message added to <strong>Detail</strong> section in DS report
Message added to <strong>Warning</strong> section in DS report
Message added to <strong>Error</strong> section in DS report
Fail to generate sequence profile from hits
When set to True, all duplicate <strong>Summary</strong> lines in the DS report are removed.
True
False
Contains less commonly used parameters.
Debug Level
Debug Message
Message to send to the log file (if log is specified)
User can set the debug level for the debug message.
1
2
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The date this component was registered.
Mon Feb 26 16:26:52 2018
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Messaging
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{8F288684-93BC-4712-BC9A-73930339BDEB}
1590 -1355
By Ref
1
100
Sends text to the client and/or to the DS report
<p>Messages can be sent to the client or text can be specified for the DS report.</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
18.1.0.1604
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
Local Component Identifier
20
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\DS Messaging
The protein sequence file(s) to read
<!-- <DSClientParameterType=AasequenceType> -->
SourceTag
Maximum
Keep Features
$(Input Sequence)
The maximum number of data records to read (all if value is empty).
1
<p>The type of tag to create to identify the source of data records. A property called <b>SourceTag</b> is created and holds the value indicated.</p>
<ul>
<li><b>None</b>: No SourceTag property is created and any existing ones are removed.</li>
<li><b>Filename</b>: The name of the file read.</li>
<li><b>FilenameAndExtension</b>: The name of the file read with its file extension.</li>
<li><b>FullFilename</b>: The name of the file read, including the directory path.</li>
</ul>
None
Filename
FilenameAndExtension
FullFilename
Read the features and annotations or just the sequence data, description, accession number and residue identifiers?
True
False
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
<p>Execute this protocol in parallel on Pipeline Pilot servers. You can set the number of data records sent in each process batch, the name of the servers, the number of protocol processes executed on each server, and whether to preserve the order of the processed data records.</p>
<p><strong>Notes</strong>:</p>
<ul>
<li>Not all parts of the protocol are run in parallel, which may affect scaling. In general, performance improves when the parallelization benefits outweigh the data transfer overhead for the specified batch size.</li>
<li>By default, Pipeline Pilot servers are configured to allow a maximum of four processes per protocol. You can adjust this on the Pipeline Pilot Server Administration Portal.</li>
</ul>
Batch Size
Server
Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
Comma separated list of servers on which to execute the remote subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog. However, specifying <b>'localhost'</b> as a server will cause the subprotocol to use its own server. This can be helpful on multi-CPU systems and Linux clusters.
Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Optional comma separated list of maximum number of batches that can simultaneously execute on each remote server. The number of entries in this parameter must equal the number of servers.
#processes := parameter();
expand(',', #processes);
if (numvalues(#processes) > 0) then
#servers := parameter('Server');
expand(',', #servers);
end if;
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ')');
numvalues(#processes) == 0 or numvalues(#processes) == numvalues(#servers);
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Protein Sequence Reader
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
DOCUMENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{D39A438D-0E20-4482-8B05-3953159EFC19}
90 -1535
By Ref
1
100
Reads protein sequence file(s)
Creates a sequence data record for each protein sequence represented in the <i>Source</i> file(s). Sequence file formats supported by Discovery Studio, like BSML, PIR, FASTA, and etc, can be used.
Internally the PIR format is used. Only features supported by that file format will be included in the data record.
None
Generic
Sequence data record(s)
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
8.5.0.200
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Mon Feb 06 13:02:46 2012
Local Component Identifier
1
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Biology\Protein Modeling\Readers\DS Protein Sequence Reader
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
use strict;
use Utilities::FileUtils;
sub onInitialize {
my $context = shift;
return pilot::READYFORINPUTDATA;
}
sub onProcess {
my $context = shift;
my $data = shift;
# fasta sequence id
my $fastaID = $data->getProperties->getHashRef->{originalFastaID};
$fastaID = $data->getProperties->getHashRef->{displayID} if(!$fastaID);
$context->getGlobalProperties->getHashRef->{SequenceName} = $fastaID;
# remove inappropriate letters for file name
$context->getGlobalProperties->getHashRef->{SequenceFileName}=Utilities::FileUtils::replaceBadFileChars($fastaID);
return pilot::READYFORINPUTDATA;
}
sub onFinalize {
my $context = shift;
}
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/dscore
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
If True, the Perl interpreter instance for the component is terminated in the onFinalize event.
True
False
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Nov 06 19:45:37 2006
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Comment
Loads a class to provide component behavior.
Component Usage
Loads a class to provide component behavior.
Set the ports and icon as appropriate.
Set SequenceName and SequenceFileName
390 -1535
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_comp_dev.pdf')">Perl Component Development Guide</a></li>
</ul>
Generic
Generic
Generic
The server version when this component was registered
6.0.1.4
Local Component Identifier
2
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
Short message send to the client.
Client Message Final
step 1/1: $(SequenceName)
Final Message reported to the DS Client
Options controlling the DS report page
Report Error
Report Summary
Remove Duplicates from Summary
Report Warning
Report Detail
Message added to <strong>Summary</strong> section in DS report
Message added to <strong>Detail</strong> section in DS report
Message added to <strong>Warning</strong> section in DS report
Message added to <strong>Error</strong> section in DS report
When set to True, all duplicate <strong>Summary</strong> lines in the DS report are removed.
True
False
Contains less commonly used parameters.
Debug Level
Debug Message
Message to send to the log file (if log is specified)
User can set the debug level for the debug message.
1
2
Temporary files converted by the network to true temporary filenames
Lists a set of properties (without the @) to create on the property list for this protocol. Other properties are created on the global property list and are accessible from all protocols.
When the network runs, processes once or until done for each process call
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The date this component was registered.
Mon Feb 26 16:26:52 2018
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
DS Messaging
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
COMPONENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{8F288684-93BC-4712-BC9A-73930339BDEB}
690 -1535
By Ref
1
100
Sends text to the client and/or to the DS report
<p>Messages can be sent to the client or text can be specified for the DS report.</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
18.1.0.1604
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
Local Component Identifier
3
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\DS Messaging
Identifies the BLAST database to use:
<ul>
<li><b>nr</b>:Non-redundant protein sequences.</li>
<li><b>refseq_protein</b>:Reference proteins.</li>
<li><b>swissprot</b>:Swissprot protein sequences.</li>
<li><b>pat</b>:Patented protein sequences.</li>
<li><b>pdb</b>:Protein Data Bank proteins.</li>
<li><b>env_nr</b>:Environmental samples.</li>
</ul>
Organism
$(Database)
An organism name, for example human or Homo sapiens, to limit search.
$(Organism)
Sets the threshold expectation value for keeping alignments. This is the E from the Karlin-Altschul equation.
$(E-value Cutoff)
Truncate the number of database hits
$(Maximum Hits)
Maximum HSPs Per Hit
Matrix
Gap Opening Penalty
Gap Extension Penalty
Word Size
Filter Query Sequence
Lower Case Filtering
Gapped Alignment
Use Composition Based Statistics
Truncate the number of HSPs (high scoring pairs) per database hit
100
Designates a protein similarity matrix
$(Scoring Matrix)
Penalty for opening a gap
$(GapOpeningPenalty)
Penalty for extending an existing gap by one character
$(GapExtensionPenalty)
Sets the word size for the initial word search
$(Word Size)
Filter the query sequence for low-complexity subsequences. Uses SEG.
$(Filter Low Complexity)
Use lower case filtering of the query sequence
True
False
Perform gapped alignment
$(Gapped Alignment)
Use composition based score adjustment.
$(Use Composition Based Statistics Flag)
Result Text Property
URL
XML Output File
Batch Search
Polling Time
Time Out
Use BioPerl BLAST Parser
If True, BioPerl will be used to parse BLAST result file. If False and <i>File Format</i> is <b>blastxml</b>, a faster proprietary parser will be used.
True
False
Name of property to hold raw result text. Property will be added to the SearchResult node.
URL for online BLAST server
$(URL)
The filename for BLAST XML output file.
$(JobDir)/$(SequenceFileName).xml
If True, queries for consecutive small sequences will be sent to NCBI together in one batch search request up to 1,000 bases. If False, each query will be sent separately.
True
False
Specifies the time in seconds to poll the job status. Do not poll more often than once a minute according to the NCBI BLAST usage guidelines.
61
Specifies the time out in minutes. If empty, the program will wait until the results come back.
60
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
An Entrez query to limit search. For example, <b>1000:2000[slen]</b> limits the search to protein entries with lengths between 1000 to 2000 residues. Refer to <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/BLAST/blastcgihelp.shtml#entrez_query')">Limit by Entrez Query</a> for more details.
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
Mismatch Penalty, Match Reward, Strand, Query Genetic Code, Database Genetic Code
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Fri Mar 6 14:41:13 2020
The component this component is based upon.
SubProtocol
{0B4076D1-858D-4C25-A621-B19D3269D770}
990 -1535
By Ref
0
100
Runs blastp online at NCBI for a protein sequence.
blastp compares incoming protein sequences with a protein database, resulting in protein alignments. This component runs blastall online at NCBI to generate BLAST results that are added to the sequence data record. A hit list data record is output for each input query sequence. The hits can be extracted from the hit list for individual manipulation using the <i>Extract BLAST Hits</i> component. Sequences with no BLAST hits are routed to the fail port.
There are parameters for the database name and for the expectation value.
The NCBI Online BLASTp component supports the following protein databases:
<table border>
<caption>Supported Online Protein Databases</caption>
<thead>
<tr>
<th>Abbreviation</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>nr</td>
<td>All non-redundant GenBank CDS translations+RefSeq Proteins+PDB+SwissProt+PIR+PRF</td>
</tr>
<tr>
<td>refseq_protein</td>
<td>Reference proteins</td>
</tr>
<tr>
<td>swissprot</td>
<td>Last major release of the SWISS-PROT protein sequence database (no updates)</td>
</tr>
<tr>
<td>pat</td>
<td>Proteins from the Patent division of GenPept</td>
</tr>
<tr>
<td>pdb</td>
<td>Sequences derived from the 3-dimensional structure from Brookhaven Protein Data Bank</td>
</tr>
<tr>
<td>env_nr</td>
<td>Metagenomic proteins</td>
</tr>
<tr>
<td>tsa_nr</td>
<td>Transcriptome Shotgun Assembly proteins</td>
</tr>
</tbody>
</table>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
21.1.0.2327
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
NCBI Online BLASTp
Protocol AutoLayout
1
0
COMPONENTNODE
Component Revision
100
Local Component Identifier
4
Subprotocol Made Link Into Copy
1
Subprotocol Modified
1
The original path within the database of this component.
Components\Biology\Sequence Analysis\Search and Similarity\NCBI BLAST\Online BLAST\NCBI Online BLASTp
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
@'Batch Search'
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Batch Search?
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -95
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
5
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
Consecutive sequences will be grouped as many as possible when their total length is smaller than the specified value.
1000
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
<p>
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
</p>
<p> </p>
<p>
Note: In an expression inside the subprotocol you can identify the category property name as:
</p>
<code>parentproperty('RunToCompletion Categorize Using')</code>
<p>
And so to access the value of this property on the current data record, use:
</p>
<code>property(parentproperty('RunToCompletion Categorize Using'))</code>
<p> </p>
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allows the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running locally or remotely. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option with care, since it will not increase throughput in all circumstances. Generally speaking, it yields improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
end if;
#parallel;
<p>
A comma-separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (that is, server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be useful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
A comma-separated list of the maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.</br></br>For non-Grid Engine job, the number of jobs will be limited to the value for the maximum number of simultaneous subprotocol jobs as defined in the Admin Console even if this value is higher than the value entered here or '*'.</br></br>For Grid Engine jobs, the number of jobs will be limited to the value for the maximum number of simultaneous subprotocol grid jobs as defined in the Admin Console even if this value is higher than the value entered here or '*'.</br></br>
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available. Note: If no grid engine is configured the subprotocol will still run even if this parameter is set to true.
Parallel Processing Options Server Run On Grid Queue Name
Parallel Processing Options Server Run On Grid Grid Options
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
<p>This is an array of command line options to send to the Grid engine submit command.</p>
<p>If you just include one line, whatever you enter will be sent "as is" to the submission command.</p>
</br>
<b>Example - Just sending options directly to command line - PBS</b>
<p>-l procs=16 -l walltime=08:00:00 -A PROJECTID</p>
</br>
<p>There are a number of parameters that Pipeline Pilot will translate to the correct command line options for whichever supported Grid Engine you are using.</p>
</br>
<b>Example - Supported parameters</b>
<p>Account=<Project/Account name></p>
<p>Maximum Execution Time=<HH:MM:SS> hours, minutes, seconds - LSF ignores seconds</p>
<p>Number of Cores=<16></p>
<p>Parallel Environment=<The Parallel Environment to use> - Only needed for SGE</p>
<p>Other=<Additional Parameters sent directly to submit command line></p>
</br>
<p>Any of these options can be commented out by placing a # in front of them.</p>
</br>
parameter('Parallel Processing Options Server Run On Grid');
Account=
Maximum Execution Time=
Parallel Environment=
Number of Cores=
Other=
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
<p>A comma-delimited list of global properties, each of which is automatically populated with the name of a file in the job's temporary folder, which always has a unique name.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold only the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Separate property names with commas. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>
<p>Any global property not declared in this way is created on the top-level global property list and is accessible from all subprotocols, but breaks the encapsulation of the subprotocol and may collide with another property of the same name.</p>
<p>Therefore, you should always declare a new global property in the <i>DeclareLocal</i> parameter, to minimize the scope of the global property to the subprotocol where it is used.</p>
<p>The example <i>DeclareLocal</i> value below creates 3 global properties in the subprotocol scope. Note that you can initialize the declared global value with a scalar value using the ":=" syntax, as shown. You can surround text string values with quotes to aid clarity.</p>
<pre>
SubP_Counter := 0, SubP_NameHash, SubP_TitleText:="Sample 1"
</pre>
TagProperty
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Internal Group Sequences by Total Length
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{B61A0ABB-9EC8-4900-86DA-8F57F6FF90A6}
390 -95
By Ref
0
100
Internal Group Sequences by Total Length
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
21.1.0.2327
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Wed Mar 4 15:31:45 2020
Local Component Identifier
4
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
OriginalPackage
scitegic/coreutils
The original path within the database of this component.
Unknown
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
#totalLength+=length;
if #totalLength>@'Total Length' then
#ndx++;
#totalLength:=length;
end if;
property(@TagProperty):='t'.#ndx;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
#ndx:=1;
#totalLength:=0;
@TagProperty:='t'.CreateGUID(1);
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Add Tag
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
90 -95
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
0
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
Al1HAAF7VGhlIG5hbWUgb2YgYSBwcm9wZXJ0eSBvciBsaXN0IG9mIHByb3BlcnRpZXMgc2VwYXJhdGVkIGJ5IGNvbW1hcyB0aGF0IGFjdCBhcyBhIGNvbW1vbiBrZXkgZm9yIHRob3NlIHJlY29yZHMgdG8gYmUgZ3JvdXBlZAosU3RyaW5nVHlwZQAMAAAAAQEAAAEcVGhpcyBwYXJhbWV0ZXIgaXMgcmVxdWlyZWQuBAH8AAAAAAAAAAAALEdyb3VwVXNpbmcDzFPPuwQA
StringType
StringType
The name of a property or list of properties separated by commas that act as a common key for those records to be grouped
GroupUsingListType
TagProperty
Indicates how "GroupUsing" properties are interpreted when more than one property is listed:
- UseFirstDefinedValue: Group using only the first property in the list with a defined value for a record.
- UseAllValues: Groups using the concatentation of the values of all properties listed; a match occurs only when records share the same value for all the properties.
UseFirstDefinedValue
UseAllValues
Contains less commonly used parameters
GroupName
Data Are Sorted
If given, a new property called "GroupName" is created with this value. It can aid in unpacking only selected groups by name.
Data
Specifies whether the input data are already sorted by the <i>GroupUsing</i> properties. If so, then the grouping can be done much faster with less memory impact. Use this option only if you are certain that the input data are sorted.
<p>
<b>Note:</b>
<ul>
<li>If you set <i>Data Are Sorted</i> to True and the input data are <b>not</b> sorted, an error is generated.
</ul>
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/generic
The date this component was registered.
Wed Mar 16 08:27:06 2011
{3FDE8197-D86F-4EB0-99FF-82D6430AF466}
Automatically update the component display name when the "GroupUsing" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
if (strlength(trim(parameter('GroupUsing', 2))) > 0) then
"Group on " . trim(parameter('GroupUsing', 2));
else
"";
end if;
JOINNODE
The component this component is based upon.
Group Data by Tag
0
390 -95
100
Groups all data records with the same property value into a single data record
<p><i>GroupUsing</i> specifies the property or comma separated list of properties to use for determining which records will be grouped together. These grouped data records can later be unbundled by <i>Ungroup Data</i>. Any records lacking the <i>GroupUsing</i> key are sent out the Fail port.</p>
<p>
One use of grouped data is to provide input to a subprotocol for which <i>RunToCompletion</i> has been set to True. The subprotocol will then operate on each group independently. You would typically use <i>Ungroup Data</i> as the first component of the subprotocol.
<p>
<b>Notes:</b>
<ul>
<li>No data are lost or changed when you create a grouped record from separate data records. Grouping is simply a way to bundle records together.
<li>For the purpose of matching records to be grouped, the <i>GroupUsing</i> property is treated as a string. (Thus, for example, the values "1.0" and "1" do not match.)
<li>The new data record has three properties defined at the top level:
<ul>
<li><b>GroupName</b> contains the name of the group as specified by the parameter <i>GroupName</i>.</li>
<li><b>NumberInGroup</b> contains the number of records in the group.
<li> The properties named by <i>GroupUsing</i> contain the values of these properties shared by all members of the group.</li>
</ul>
</ul>
<b>See Also:</b>
<ul>
<li>To group every N data records together as an alternative to using a <i>GroupUsing</i> property, use <a href="javascript:DoLink('{C3A203C2-9470-47F8-A7FF-D5C6DD1BAA67}')">Group Data by Number</a>
<li>To separate the grouped data into the original records, use <a href="javascript:DoLink('{57833F08-BD53-42AE-A14C-87EED61B8FBD}')">Ungroup Data</a>. You would typically use Ungroup Data as the first component of a <i>RunToCompletion</i> subprotocol designed to operate on grouped data.
<li>For a different approach to combining data records, analogous to a database join operation, see <a href="javascript:DoLink('{1E393621-138C-466E-8A2B-275A3FAB26E3}')">Merge Data</a>.
</ul>
Generic
Generic
Data records, each containing a collection of grouped records stored in a hierarchy
Generic
Data records that lack the <i>GroupUsing</i> property
The server version when this component was registered
8.4.5.20
Local Component Identifier
1
The original path within the database of this component.
Components\Data Access and Manipulation\Data Manipulators\Group Data by Tag
An XPath query or queries to determine the nodes to apply the PilotScript to. For example, "//*[@ID]" would select all nodes with the property "ID" defined; if the Expression were "NewID := 'NewID_' . ID;", then a new property NewId would be created, and consist of the old ID prepended with the string NewId_.
/SciTegic.data.GroupNode/*
Expression (or list of expressions separated by semi-colons) to evaluate for each selected node
Initial Expression
Final Expression
remove(@TagProperty);
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
The name of the parser to use for the query. Depending on the choice of parser, different XPath features will be supported. Primarily, "Simple XPath" is fast but supports fewer features, and "Full XPath" is show but supports more XPath features.
Simple XPath
Full XPath
When a General or DataType error occurs during processing:
LeaveData: The data is passed unalterd
ClearData: The data is cleared before being passed (an empty data item)
DeleteData: The data is deleted and not passed
LeaveData
ClearData
DeleteData
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Apply PilotScript
<p>
Our view of the data record for use in the Select query.
</p>
Attributes
Elements
ComponentNode
The user who registered this component.
scitegic/generic
The date this component was registered.
Fri Aug 17 08:38:14 2012
Remove Tag
0
{AEE34E57-CBE3-4091-AE4A-2AF22B7CF406}
690 -95
100
Selects a set of nodes in the data record and applies the PilotScript expression to its properties.
Selects a set of nodes in the data record and applies a PilotScript expression to each.
For example, "//*[@ID]" would select all nodes with the property "ID" defined; if the Expression were "NewID := 'NewID_' . ID;", then a new property NewId would be created, and consist of the old ID prepended with the string NewId_.
Generic
Generic
None
Local Component Identifier
2
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Data Tree Manipulators\Apply PilotScript to Selected Nodes
9.0.0.8
blastp
Database
UsingParent
Identifies the BLAST database to use:
<ul>
<li><b>nr</b>: Nucleotide collection.</li>
<li><b>refseq_rna</b>: Reference mRNA sequences.</li>
<li><b>refseq_genomic</b>: Reference genomic sequences.</li>
<li><b>chromosome</b>: NCBI Genomes.</li>
<li><b>est</b>: Expressed sequence tags.</li>
<li><b>est_human</b>: Human subset of est.</li>
<li><b>est_mouse</b>: Mouse subset of est.</li>
<li><b>est_others</b>: Non-human, non-mouse ESTs.</li>
<li><b>gss</b>: Genomic survey sequences.</li>
<li><b>htgs</b>: High throughput genomic sequences.</li>
<li><b>pat</b>: Patent sequences.</li>
<li><b>pdb</b>: Protein Data Bank.</li>
<li><b>alu</b>: Human ALU repeat elements.</li>
<li><b>dbsts</b>: Sequence tagged sites.</li>
<li><b>wgs</b>: Whole-genome shotgun reads.</li>
<li><b>env_nt</b>: Environmental samples.</li>
</ul>
Organism
Entrez Query
$(Database)
Organism
An organism name, for example human or Homo sapiens, to limit search.
$(Organism)
Entrez Query
An Entrez query to limit search. For example, <b>1000:2000[slen]</b> limits the search to protein entries with lengths between 1000 to 2000 residues. Refer to <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/BLAST/blastcgihelp.shtml#entrez_query')">Limit by Entrez Query</a> for more details.
$(Entrez Query)
Expectation Value
UsingParent
Sets the threshold expectation value for keeping alignments. This is the E from the Karlin-Altschul equation. Scientific notation may be used, e.g., 1e-3 or 1.5e-3.
$(Expectation Value)
Maximum Hits
UsingParent
Truncate the number of database hits
$(Maximum Hits)
UsingParent
Search Options
Database Genetic Code
Query Genetic Code
Use Composition Based Statistics
Matrix
Gapped Alignment
Lower Case Filtering
Filter Query Sequence
Strand
Word Size
Match Reward
Mismatch Penalty
Gap Extension Penalty
Gap Opening Penalty
Maximum HSPs Per Hit
Maximum HSPs Per Hit
UsingParent
Truncate the number of HSPs (high scoring pairs) per database hit
$(Maximum HSPs Per Hit)
Gap Opening Penalty
UsingParent
Penalty for opening a gap
$(Gap Opening Penalty)
Gap Extension Penalty
UsingParent
Penalty for extending an existing gap by one character
$(Gap Extension Penalty)
Mismatch Penalty
UsingParent
Sets the penalty for a nucleotide mismatch
$(Mismatch Penalty)
Match Reward
UsingParent
Sets the reward for a nucleotide match
$(Match Reward)
Word Size
UsingParent
Sets the word size for the initial word search
$(Word Size)
Strand
UsingParent
Chooses which strand of DNA-based queries is searched
$(Strand)
Filter Query Sequence
UsingParent
Filter the query sequence for low-complexity subsequences. Uses DUST.
$(Filter Query Sequence)
Lower Case Filtering
UsingParent
Use lower case filtering of the query sequence
$(Lower Case Filtering)
Gapped Alignment
UsingParent
Perform gapped alignment
$(Gapped Alignment)
UsingParent
Matrix
Designates a protein similarity matrix
$(Matrix)
Use Composition Based Statistics
UsingParent
Use composition based score adjustment.
$(Use Composition Based Statistics)
Query Genetic Code
UsingParent
Genetic code to use for translation of the query nucleotide sequence
$(Query Genetic Code)
Database Genetic Code
UsingParent
Genetic code to use for translation of the database nucleotide sequence
$(Database Genetic Code)
Additional Options
UsingParent
Result Text Property
URL
XML Output File
Polling Time
Time Out
Use BioPerl BLAST Parser
Result Text Property
UsingParent
Name of property to hold raw result text. Property will be added to the SearchResult node.
$(Result Text Property)
URL
UsingParent
$(URL)
XML Output File
UsingParent
$(XML Output File)
Specifies the time in seconds to poll the job status. Do not poll more often than once a minute according to the NCBI BLAST usage guidelines.
$(Polling Time)
Specifies the time out in minutes. If empty, the program will wait until the results come back.
$(Time Out)
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allows the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running locally or remotely. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option with care, since it will not increase throughput in all circumstances. Generally speaking, it yields improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
end if;
#parallel;
<p>
A comma-separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (that is, server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be useful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
A comma-separated list of the maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
Parallel Processing Options Server Run On Grid Grid Options
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Use this to specify grid engine specific command line parameters to the job submission command. Leave blank to use defaults.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
<p>A comma-delimited list of global properties, each of which is automatically populated with the name of a file in the job's temporary folder, which always has a unique name.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold only the leaf file name of the temporary file.</p>
responseXML
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Separate property names with commas. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>
<p>Any global property not declared in this way is created on the top-level global property list and is accessible from all subprotocols, but breaks the encapsulation of the subprotocol and may collide with another property of the same name.</p>
<p>Therefore, you should always declare a new global property in the <i>DeclareLocal</i> parameter, to minimize the scope of the global property to the subprotocol where it is used.</p>
<p>The example <i>DeclareLocal</i> value below creates 3 global properties in the subprotocol scope. Note that you can initialize the declared global value with a scalar value using the ":=" syntax, as shown. You can surround text string values with quotes to aid clarity.</p>
<pre>
SubP_Counter := 0, SubP_NameHash, SubP_TitleText:="Sample 1"
</pre>
resultFound, encodedQuery, RID, StartTime, BlastError, SeqCount, SeqCache
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Use BioPerl BLAST Parser
UsingParent
Al1HAAKZAElmIFRydWUsIEJpb1Blcmwgd2lsbCBiZSB1c2VkIHRvIHBhcnNlIEJMQVNUIHJlc3VsdCBmaWxlLiBJZiBGYWxzZSBhbmQgPGk+RmlsZSBGb3JtYXQ8L2k+IGlzIDxiPmJsYXN0eG1sPC9iPiwgYSBmYXN0ZXIgcHJvcHJpZXRhcnkgcGFyc2VyIHdpbGwgYmUgdXNlZC4gJEJvb2xUeXBlBAgAAoYAU2V0RGlzYWJsZWREZXRhaWxzKAoJICAgICAgICdUbyBlbmFibGUgJy5QYXJhbWV0ZXJOYW1lKCkuJywgc2V0IEZpbGUgRm9ybWF0IHRvICJibGFzdHhtbCInKTsKCVBhcmFtZXRlcignRmlsZSBGb3JtYXQnKSBlcSAnYmxhc3R4bWwnOwAAAQQBAAQA/AAAAAAIFFRydWUYRmFsc2UIFFRydWUYRmFsc2UAAQFkAv5qAQAAARlVc2UgQmlvUGVybCBCTEFTVCBQYXJzZXIDzFPPuxRUcnVlAwAAACAIARFTQ0lfUHJvbW90ZWRGcm9tAx09w44EARlVc2UgQmlvUGVybCBCTEFTVCBQYXJzZXIBGFNDSV9Qcm9tb3Rpb25SZW5hbWVSdWxlA8xTz7swVXNpbmdQYXJlbnQ=
BoolType
LinkGlobalPropertyType
If True, BioPerl will be used to parse BLAST result file. If False and <i>File Format</i> is <b>blastxml</b>, a faster proprietary parser will be used.
SetDisabledDetails(
'To enable '.ParameterName().', set File Format to "blastxml"');
Parameter('File Format') eq 'blastxml';
True
False
Use BioPerl BLAST Parser
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Internal Remote BLAST
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{866CC28A-2457-4ECF-A762-552D4E4FE7A6}
690 -275
By Ref
0
100
Internal Remote BLAST
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
21.1.0.2327
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Thu Mar 5 09:00:05 2020
Local Component Identifier
2
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
OriginalPackage
scitegic/seqanal
The original path within the database of this component.
Components\Biology\Sequence Analysis\Utilities\Internals\Internal Remote BLAST
Al1HAALYAFRoZSBuYW1lIG9mIHRoZSBjYWNoZSB0byB3cml0ZSwgZm9yIGV4YW1wbGU6IEEsIDEyMzQgb3IgTXlDYWNoZS4gIFRvIGNyZWF0ZSBhIHRlbXBvcmFyeSBjYWNoZUlEIHVzZSB0aGUgPGEgaHJlZj0iamF2YXNjcmlwdDpEb0xpbmsoJ3tCOTFFRTUyMS1DMDI4LTQ2QkUtODlBNC01MUNERTk4Nzc1ODl9JykiPkNyZWF0ZSBUZW1wb3JhcnkgQ2FjaGVJRHM8L2E+IGNvbXBvbmVudC4KLFN0cmluZ1R5cGUADAAAAAEBAAEABAD8AAAAAAAAAAAAIENhY2hlSUQDzFPPuyRTZXFDYWNoZQA=
StringType
StringType
The name of the cache to write, for example: A, 1234 or MyCache. To create a temporary cacheID use the <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a> component.
Maximum
Append
Scope
SeqCache
The maximum number of data records to write to the cache.
If True, append to a currently-existing cache if possible; if False, clear any current contents of the cache
True
False
The scope of the cache determines the length of time the cache exists and
the visibility of the cache to other jobs and users. <I>Job Only</I> creates a cache that is visible
only to components in the currently-running protocol, and which is deleted when the protocol finishes.
<I>User Only</I> creates a cache that is visible to components in any protocol a particular user runs;
<I>All Users</I> creates a cache that is visible to any user on the Pipeline Pilot server.
<P>Each scope has its own namespace, so you have to
specify both the <i>CacheID</i> and the <i>Scope</i> to fully determine a particular cache.</P>
<P>Since the caches created with scopes <I>User Only</I> and <I>All Users</I>
may be accessed by multiple jobs, you should use caution, as problems may arise if two jobs try to write to
the same shared cache at once. Also, to save disk space, you are responsible for deleting these caches
when finished, using the <I>Delete Cache</I> component. </P>
Job Only
User Only
All Users
If set to True, creates an index for the cache that can be used by subsequent <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a> components. As long as the later Join components use identical settings, the created index will greatly improve the performance of the join. When set to true, you will also need to update <i>JoinUsing</i>, <i>JoinUsingListType</i>, <i>JoinPropertyType</i>, and <i>IgnoreCase</i> to match your Join component.
JoinUsing
IgnoreCase
JoinPropertyType
JoinUsingListType
True
False
The name of the property or properties to index for use by <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a> components. This parameter must match the identically named parameter on the Join component in order to benefit from the created index.
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
Parameter('Create Index for Join');
<p>Indicates how <i>JoinUsing</i> properties are interpreted for creating a cache index when more than one property is listed:</p>
<ul>
<li><i>UseFirstDefinedValue</i>: The join will use only the first property in the list with a defined value for a record.
<li><i>UseAllValues</i>: The join will use the concatentation of the values of all properties listed; a match occurs only when records share the same value for all the properties.
</ul>
<p>This parameter must match the identically named parameter on the Join component in order to benefit from the created index.</p>
Parameter('Create Index for Join');
UseFirstDefinedValue
UseAllValues
The data type of the <i>JoinUsing</i> property. This parameter must match the identically named parameter on the Join component in order to benefit from the created index.
Parameter('Create Index for Join');
IsNumeric
IsText
IsDate
Consider lower-case and upper-case letters as equivalent when determining the identity of <i>JoinUsing</i> property values. This parameter must match the identically named parameter on the Join component in order to benefit from the created index.
Parameter('Create Index for Join');
True
False
Additional parameters to control the component behavior.
Write Globals
Enable Compression
<p>Specifies which globals, if any, are written into the cache.</p>
<p>The value is an array of top-level global names or global nodes specified using XPath notation. Note that node names should end with a forward slash (/).</p>
<p>Specifies if compression of the data should be enabled whenever available.</p>
<b>Warning:<b> Due to the resulting larger size of the cache files when using uncompressed files, this option should typically only be used for smaller data sets.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The cache type, that determines its folder location within the job.
Specifies the action of this component:
- Writer: The component writes data records to a new cache.
- Reader: The component reads data records from a pre-existing, completed cache.
- Writer/Reader: The component writes data out to a cache and then reads it in again to generate new records.
Writer
Reader
Writer/Reader
The user who registered this component.
scitegic/generic
The date this component was registered.
Wed Oct 03 14:48:02 2012
Cache Writer
1365 -95
DATABASENODE
{674E3781-33EA-42BB-9099-49CB4C3D08E6}
The component this component is based upon.
Cache Writer
0
100
Writes data records to a named cache
<p>A data cache is place where you can temporarily store data records without losing information. Use <i>CacheID</i> to specify the name of the cache. A cache can contain records of different types.</p>
<b>Notes:</b>
<ul>
<li>To avoid clashes with other cache names, you can use the <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a> component to obtain a global with a unique cacheID. For example, you could have a temporary cacheID identified by MYCACHE and set <i>CacheID</i> to $(MYCACHE).</li>
<li>Typically, a cache is scoped to only be visible to component in the currently-running job. To create caches that outlive a job, and may be seen by other users, use the <I>Scope</I> parameter.</li>
<li>Caches can be used with the <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a> component to add properties to a separate data stream. To improve the performance of the join, the cache can be indexed during creation by using the <i>Create Index for Join</i> parameter.</li>
</ul>
<b>See Also:</b>
<ul>
<li>To retrieve data from a cache, use <a href="javascript:DoLink('{704E3781-33EA-42BB-9099-05B44C3D08E6}')">Cache Reader</a>.</li>
<li>To join against a cache, use <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a>.</li>
<li>To delete a cache, use <a href="javascript:DoLink('{704E3781-33EA-42BB-9088-05B44C3D08E6}')">Delete Cache</a>.</li>
</ul>
Generic
None
None
The server version when this component was registered
9.0.0.100
Local Component Identifier
0
The original path within the database of this component.
Components\Data Access and Manipulation\File Writers\Cache Writer
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@resultFound:=0;
@EncodedQuery:='';
@BlastError:='';
@StartTime:=DateSecondsSince1970();
@SeqCache:='s'.CreateGUID(1);
@SeqCount:=0;
@'Result Text Property':=Trim(@'Result Text Property');
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Initialize Globals
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
90 -275
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
16
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
Contains less commonly used parameters
GroupName
(Optional) Restricts the ungroup process to only those groups having the given name -- the same name as specified by the "Group Data" component
Data
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
The user who registered this component.
scitegic/generic
The date this component was registered.
Thu Jan 19 10:40:59 2006
Ungroup Data
GENERATORNODE
The component this component is based upon.
Ungroup Data
{57833F08-BD53-42AE-A14C-87EED61B8FBD}
390 -275
0
100
Separates data records previously grouped together into the original records
<p>The input record is assumed to be a grouped data record. The individual records are extracted from the group and sent out the Pass port. The incoming data record is discarded.</p>
<p>You would typically use Ungroup Data as the first component inside a <i>RunToCompletion</i> subprotocol that expects grouped data. This executes the remainder of the subprotocol over the individual data records in the group.</p>
<b>Note:</b>
<ul>
<li>It is possible to have a group which in turn has another group as one of its members (nested groups). In this case, ungrouping will separate only the top level groups. Multiple Ungroup Data components operating in series are needed to separate nested groups.</li>
</ul>
<b>See Also:</b>
<ul>
<li>To group data records together use <a href="javascript:DoLink('{3FDE8197-D86F-4EB0-99FF-82D6430AF466}')">Group Data by Tag</a> or <a href="javascript:DoLink('{C3A203C2-9470-47F8-A7FF-D5C6DD1BAA67}')">Group Data by Number</a>.</li>
<li>To view the structure of grouped data records use <a href="javascript:DoLink('{AA4369ED-A1A1-4949-8520-39B23096ACFD}')">Data Record Tree Viewer</a></li>
<li>To learn more about <i>RunToCompletion</i> subprotocols, see the parameter help on <a href="javascript:DoLink('{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}')">Subprotocol</a>.</li>
</ul>
Generic Data
Data records with a hierarchical group of other data records
Generic Data
The individual data records that were contained within a hierarchical group of the incoming data
Generic Data
Data records on which the GroupName property is missing
The server version when this component was registered
6.0.0.6
Local Component Identifier
18
The original path within the database of this component.
Components\Data Access and Manipulation\Data Manipulators\Ungroup Data
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
use strict;
use SciTegic::Bio::Tools::Blast::BlastUtilities;
use SciTegic::Bio::PipelinePilot::ToBioPerl::CreateBioPerlRichSequence;
use SciTegic::Bio::PipelinePilot::Validate::SequenceValidation;
my $requiredAlphabet;
sub onInitialize {
my $context = shift;
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $program = $globalsHashRef->{"Program"};
$requiredAlphabet = getRequiredAlphabetForProgram($program);
return pilot::READYFORINPUTDATA;
}
sub onProcess {
my $context = shift;
my $data = shift;
my $propertiesHashRef = $data->getProperties->getHashRef();
my $sequenceNode = $data->getRoot();
if (!isSequence($sequenceNode))
{
$data->throwDataTypeError("Sequence");
}
if (!sequenceMatchesRequiredAlphabet($sequenceNode, $requiredAlphabet))
{
die("Sequence alphabet and BLAST program mismatch\n");
}
my $seq = createBioPerlRichSequence($context, $sequenceNode);
my $queryHeader = ">".(defined $seq->display_id() ? $seq->display_id() : "").
" ".(defined $seq->desc() ? $seq->desc() : "");
$propertiesHashRef->{'queryHeader'} = $queryHeader;
return pilot::READYFORINPUTDATA;
}
sub onFinalize {
my $context = shift;
}
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/seqanal
<p>
The Perl component functions by creating an instance of a Perl interpreter and
passing the supplied Perl script to the interpreter for execution. The
creation of the Perl interpreter occurs one time, while script execution
occurs once for initialization and finalization subroutines and once
for each data record processed by the component.
</p>
<p>
However, in the context of a Run To Completion (RTC) subprtocols, the Perl
component would normally be re-instantiated for each data record,
in common with other components in an RTC subprotocol. This implies the
re-creation of the Perl interpreter for each data record, which can be
relatively time consuming, particularly for Perl scripts which
themselves perform a rapid operation per data record.
</p>
<p>
So, to avoid significant differences in performance timings between RTC
and non-RTC contexts, the Perl component's default behavior is to <i>keep</i>
the same instance of the Perl interpreter throughout its life cycle. This
is indicated by the False setting of the <em>Rebuild In RTC</em> parameter.
</p>
<p>
However, there are some circumstances where Perl script or 3rd party modules
are not written in a way that allows them to safely re-use the same interpreter in this way. So if you see unexpected behavior from a Perl
component in a Run To Completion subprotocol, switch the <em>Rebuild In RTC</em>
parameter to True to see if that resolves the issue.
</p>
<p>
If that fixes the problem, and the Perl cannot be re-cast to
handle the persistent interpreter,
you may need to trade performance for stability.
</p>
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Tue Feb 05 15:52:10 2013
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Perl (on Server)
690 -275
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
<li> If you see unexpected or erroneous behavior when using this component
in a Run To Completion subprotocol, please look at the help text for the
<em>Rebuild In RTC</em> parameter, which is in the Implementation tab.</li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/compdev_perl.pdf')">Perl Component Development Guide</a></li>
</ul>
Generic
Generic
Generic
The server version when this component was registered
9.0.1.15
Local Component Identifier
19
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@SeqCount++;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Sequence Count
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1065 -275
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
33
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
// Encode Ctrl-A characters in description
if queryHeader rlike '\cA' then
queryHeader := rsubst(queryHeader, '\cA', '%01', 'g');
end if;
@EncodedQuery.= URLEncode(queryHeader).'%0A'.data.'%0A';
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Encode Query
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1365 -455
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
22
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
StringType
Identifies the BLAST database to use:
<ul>
<li><b>nr</b>: Nucleotide collection.</li>
<li><b>refseq_rna</b>: Reference mRNA sequences.</li>
<li><b>refseq_genomic</b>: Reference genomic sequences.</li>
<li><b>chromosome</b>: NCBI Genomes.</li>
<li><b>est</b>: Expressed sequence tags.</li>
<li><b>est_human</b>: Human subset of est.</li>
<li><b>est_mouse</b>: Mouse subset of est.</li>
<li><b>est_others</b>: Non-human, non-mouse ESTs.</li>
<li><b>gss</b>: Genomic survey sequences.</li>
<li><b>htgs</b>: High throughput genomic sequences.</li>
<li><b>pat</b>: Patent sequences.</li>
<li><b>pdb</b>: Protein Data Bank.</li>
<li><b>alu</b>: Human ALU repeat elements.</li>
<li><b>dbsts</b>: Sequence tagged sites.</li>
<li><b>wgs</b>: Whole-genome shotgun reads.</li>
<li><b>env_nt</b>: Environmental samples.</li>
</ul>
Organism
Entrez Query
Database
StringType
Sets the threshold expectation value for keeping alignments. This is the E from the Karlin-Altschul equation. Scientific notation may be used, e.g., 1e-3 or 1.5e-3.
Expectation Value
StringType
Truncate the number of database hits
Maximum Hits
GroupType
Maximum HSPs Per Hit
Gap Opening Penalty
Gap Extension Penalty
Mismatch Penalty
Match Reward
Word Size
Strand
Filter Query Sequence
Lower Case Filtering
Gapped Alignment
Matrix
Use Composition Based Statistics
Query Genetic Code
Database Genetic Code
Search Options
StringType
Truncate the number of HSPs (high scoring pairs) per database hit
Maximum HSPs Per Hit
StringType
Penalty for opening a gap
Gap Opening Penalty
StringType
Penalty for extending an existing gap by one character
Gap Extension Penalty
StringType
Sets the penalty for a nucleotide mismatch
Mismatch Penalty
StringType
Sets the reward for a nucleotide match
Match Reward
StringType
Sets the word size for the initial word search
Word Size
StringType
Chooses which strand of DNA-based queries is searched
Strand
StringType
Filter the query sequence for low-complexity subsequences. Uses DUST.
Filter Query Sequence
StringType
Use lower case filtering of the query sequence
Lower Case Filtering
StringType
Perform gapped alignment
Gapped Alignment
StringType
Designates a protein similarity matrix
Matrix
StringType
Use composition based score adjustment.
Use Composition Based Statistics
StringType
Genetic code to use for translation of the query nucleotide sequence
Query Genetic Code
StringType
Genetic code to use for translation of the database nucleotide sequence
Database Genetic Code
GroupType
Result Text Property
URL
XML Output File
Additional Options
StringType
Name of property to hold raw result text. Property will be added to the SearchResult node.
Result Text Property
StringType
URL for online BLAST server
URL
StringType
The filename for BLAST XML output file.
XML Output File
Program
$(Program)
Define onInitialScript, onScript and onFinalScript subroutines here.
package main;
#
# RemoteBlastFormatQuery.pm
#
# This program and all files, classes and data used by it
# are protected by Copyright and hence may not be used, copied,
# modified, transmitted, inspected, or executed by any means including
# the use of electronic data processing equipment, xerography, or
# any other methods without the express written permission of the
# copyright holder.
#
# Copyright (C) 2017 Accelrys Software Inc., All Rights Reserved
#
use strict;
use SciTegic::Bio::Tools::Blast::BlastUtilities;
use SciTegic::Bio::PipelinePilot::Utils::PipelinePilotUtilities;
use URI::Escape;
my $debugMode = 0;
sub onInitialize {
my $context = shift;
$debugMode = $context->getGlobalProperties()->getHashRef()->{"debugMode"}=~/true/i ? 1 : 0;
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $program = $parametersHashRef->{"Program"};
my $database = $parametersHashRef->{"Database"};
my $url = $parametersHashRef->{"URL"};
my $service = $parametersHashRef->{"Service"};
$::maximumHSPsPerHit = $parametersHashRef->{"Maximum HSPs Per Hit"};
$::resultTextProperty = $parametersHashRef->{"Result Text Property"};
$::requestBody = '';
$::requestBody = setRemoteBlastParameters($parametersHashRef);
$::requestBody .= '&PROGRAM=' . $program . '&DATABASE=' . $database;
if ($service ne ''){
$::requestBody .= '&SERVICE=$service';
} else {
$::requestBody .= '&SERVICE=plain';
}
$::blastOutputFileName = $parametersHashRef->{"XML Output File"};
$::blastOutputFileName=~s/^\s+//;
$::blastOutputFileName=~s/\s$//;
return pilot::READYFORINPUTDATA;
}
sub onProcess {
my $context = shift;
my $data = shift;
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $propertiesHashRef = $data->getProperties->getHashRef();
$propertiesHashRef->{'requestBody'} = $::requestBody.'&QUERY='.$globalsHashRef->{'EncodedQuery'};
$context->debugMessage("request is " . $propertiesHashRef->{'requestBody'});
return pilot::READYFORINPUTDATA;
}
sub onFinalize {
my $context = shift;
}
sub setRemoteBlastParameters
{
my ($parametersHashRef) = @_;
my @parameters = loadBlastParameters($parametersHashRef);
my %parametersHash = @parameters;
$parametersHash{'Organism'} = $parametersHashRef->{'Organism'};
$parametersHash{'Entrez Query'} = $parametersHashRef->{'Entrez Query'};
setRemoteBlastHeader(\%parametersHash);
$::requestBody =~ s/^&//;
return $::requestBody;
}
sub setRemoteBlastHeader
{
my ($parametersHashRef) = @_;
if ($parametersHashRef->{"e"})
{
$::requestBody .= '&EXPECT=' . $parametersHashRef->{"e"};
}
if ($parametersHashRef->{"b"})
{
$::requestBody .= '&ALIGNMENTS=' . $parametersHashRef->{"b"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"ALIGNMENTS"} = $parametersHashRef->{"b"};
# $Bio::Tools::Run::RemoteBlast::RETRIEVALHEADER{"ALIGNMENTS"} = $parametersHashRef->{"b"};
}
if ($parametersHashRef->{"v"})
{
$::requestBody .= '&DESCRIPTIONS=' . $parametersHashRef->{"v"}
# $Bio::Tools::Run::RemoteBlast::HEADER{"DESCRIPTIONS"} = $parametersHashRef->{"v"};
# $Bio::Tools::Run::RemoteBlast::RETRIEVALHEADER{"DESCRIPTIONS"} = $parametersHashRef->{"v"};
}
if ($parametersHashRef->{"M"})
{
$::requestBody .= '&MATRIX_NAME=' . $parametersHashRef->{"M"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"MATRIX_NAME"} = $parametersHashRef->{"M"};
}
if ($parametersHashRef->{"G"} && $parametersHashRef->{"E"})
{
$::requestBody .= '&GAPCOSTS=' . $parametersHashRef->{"G"} . " " . $parametersHashRef->{"E"};
# $::requestBody .= '&GAPCOSTS=' . uri_escape($parametersHashRef->{"G"} . " " . $parametersHashRef->{"E"});
# $Bio::Tools::Run::RemoteBlast::HEADER{"GAPCOSTS"} = $parametersHashRef->{"G"} . " " . $parametersHashRef->{"E"};
}
if ($parametersHashRef->{"W"})
{
$::requestBody .= '&WORD_SIZE=' . $parametersHashRef->{"W"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"WORD_SIZE"} = $parametersHashRef->{"W"};
}
if ($parametersHashRef->{"F"})
{
if ($parametersHashRef->{"F"} eq "T")
{
$::requestBody .= '&FILTER=L';
# $Bio::Tools::Run::RemoteBlast::HEADER{"FILTER"} = "L";
}
elsif ($parametersHashRef->{"F"} eq "F")
{
$::requestBody .= '&FILTER=F';
# delete($Bio::Tools::Run::RemoteBlast::HEADER{"FILTER"});
}
}
if ($parametersHashRef->{"U"})
{
my $lowerCaseFilteringValue = "yes";
if ($parametersHashRef->{"U"} eq "F")
{
$lowerCaseFilteringValue = "no";
}
$::requestBody .= '&LCASE_MASK=' . $lowerCaseFilteringValue;
# $Bio::Tools::Run::RemoteBlast::HEADER{"LCASE_MASK"} = $lowerCaseFilteringValue;
}
if ($parametersHashRef->{"g"})
{
my $ungappedAlignmentValue = "yes";
if ($parametersHashRef->{"g"} eq "T")
{
$ungappedAlignmentValue = "no";
}
$::requestBody .= '&UNGAPPED_ALIGNMENT=' . $ungappedAlignmentValue;
# $Bio::Tools::Run::RemoteBlast::HEADER{"UNGAPPED_ALIGNMENT"} = $ungappedAlignmentValue;
}
if ($parametersHashRef->{"q"})
{
$::requestBody .= '&NUCL_PENALTY=' . $parametersHashRef->{"q"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"NUCL_PENALTY"} = $parametersHashRef->{"q"};
}
if ($parametersHashRef->{"r"})
{
$::requestBody .= '&NUCL_REWARD=' . $parametersHashRef->{"r"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"NUCL_REWARD"} = $parametersHashRef->{"r"};
}
if ($parametersHashRef->{"Q"})
{
$::requestBody .= '&GENETIC_CODE=' . $parametersHashRef->{"Q"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"GENETIC_CODE"} = $parametersHashRef->{"Q"};
}
if ($parametersHashRef->{"D"})
{
$::requestBody .= '&DB_GENETIC_CODE=' . $parametersHashRef->{"D"};
# $Bio::Tools::Run::RemoteBlast::HEADER{"DB_GENETIC_CODE"} = $parametersHashRef->{"D"};
}
if ($parametersHashRef->{"t"})
{
if ($parametersHashRef->{"t"} eq "T")
{
$::requestBody .= '&COMPOSITION_BASED_STATISTICS=1';
# $Bio::Tools::Run::RemoteBlast::HEADER{'COMPOSITION_BASED_STATISTICS'} = "1";
}
elsif ($parametersHashRef->{"t"} eq "F")
{
$::requestBody .= '&COMPOSITION_BASED_STATISTICS=0';
# $Bio::Tools::Run::RemoteBlast::HEADER{'COMPOSITION_BASED_STATISTICS'} = "0";
}
}
# Limit by Entrez Query
my $organism = $parametersHashRef->{'Organism'};
$organism=~s/^\s+//;
$organism=~s/\s$//;
$organism .= '[orgn] ' if(length($organism));
my $entrezQuery = $parametersHashRef->{'Entrez Query'};
$entrezQuery=~s/^\s+//;
$entrezQuery=~s/\s+$//;
if(length($entrezQuery) > 0) {
$entrezQuery = $organism.' AND '.$entrezQuery if(length($organism)>0);
} else {
$entrezQuery = $organism;
}
$::requestBody .= '&ENTREZ_QUERY=' . $entrezQuery if(length($entrezQuery));
# $Bio::Tools::Run::RemoteBlast::HEADER{'ENTREZ_QUERY'} = $entrezQuery if(length($entrezQuery));
# XML output format
$::requestBody .= '&FORMAT_TYPE=XML&CMD=Put&ALIGNMENT_VIEW=Pairwise&FORMAT_OBJECT=Alignment';
#$Bio::Tools::Run::RemoteBlast::RETRIEVALHEADER{"FORMAT_TYPE"} = 'XML';
}
1;
scitegic/seqanal
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
An organism name, for example human or Homo sapiens, to limit search.
Organism
An Entrez query to limit search. For example, <b>1000:2000[slen]</b> limits the search to protein entries with lengths between 1000 to 2000 residues. Refer to <a href="javascript:DoLink('http://www.ncbi.nlm.nih.gov/BLAST/blastcgihelp.shtml#entrez_query')">Limit by Entrez Query</a> for more details.
Entrez Query
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Fri Nov 21 09:11:28 2008
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Set BLAST Parameters
390 -635
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Runs blastn online at NCBI for a nucleotide sequence.
blastn compares incoming nucleotide sequences with a nucleotide database, resulting in nucleotide alignments. This component runs blastall online at NCBI to generate BLAST results that are added to the sequence data record. . A hit list data record is output for each input query sequence. The hits can be extracted from the hit list for individual manipulation using the <i>Extract BLAST Hits</i> component. Sequences with no BLAST hits are routed to the fail port.
There are parameters for the database name and for the expectation value.
The NCBI Online BLASTn component supports the following nucleotide databases:
<table border>
<caption>Supported Online Nucleotide Databases</caption>
<thead>
<tr>
<th>Abbreviation</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>nr</td>
<td>All GenBank+RefSeq Nucleotides+EMBL+DDBJ+PDB sequences (but no EST, STS, GSS, or phase 0, 1 or 2 HTGS sequences). No longer "non-redundant"</td>
</tr>
<tr>
<td>refseq_rna</td>
<td>RNA entries from NCBI's Reference Sequence project</td>
</tr>
<tr>
<td>refseq_genomic</td>
<td>Genomic entries from NCBI's Reference Sequence project</td>
</tr>
<tr>
<td>chromosome</td>
<td>Searches Complete Genomes, Complete Chromosome, or contigs form the NCBI Reference Sequence project</td>
</tr>
<tr>
<td>est</td>
<td>Database of GenBank+EMBL+DDBJ sequences from EST Divisions</td>
</tr>
<tr>
<td>est_human</td>
<td>Human subset of GenBank+EMBL+DDBJ sequences from EST Divisions</td>
</tr>
<tr>
<td>est_mouse</td>
<td>Mouse subset of GenBank+EMBL+DDBJ sequences from EST Divisions</td>
</tr>
<tr>
<td>est_others</td>
<td>Non-Mouse, non-Human sequences of GenBank+EMBL+DDBJ sequences from EST Divisions</td>
</tr>
<tr>
<td>gss</td>
<td>Genome Survey Sequence, includes single-pass genomic data, exon-trapped sequences, and Alu PCR sequences</td>
</tr>
<tr>
<td>htgs</td>
<td>Unfinished High Throughput Genomic Sequences: phases 0, 1 and 2 (finished, phase 3 HTG sequences are in nr)</td>
</tr>
<tr>
<td>pat</td>
<td>Nucleotides from the Patent division of GenBank</td>
</tr>
<tr>
<td>pdb</td>
<td>Sequences derived from the 3-dimensional structure from Brookhaven Protein Data Bank</td>
</tr>
<tr>
<td>alu</td>
<td>Select Alu repeats from REPBASE, suitable for masking Alu repeats from query sequences</td>
</tr>
<tr>
<td>dbsts</td>
<td>Database of GenBank+EMBL+DDBJ sequences from STS Divisions</td>
</tr>
<tr>
<td>wgs</td>
<td>Whole genome shotgun sequence entries</td>
</tr>
<tr>
<td>env_nt</td>
<td>Nucleotide sequences from environmental samples, including those from Sargasso Sea and Mine Drainage projects</td>
</tr>
</tbody>
</table>
Generic
Generic
Data records which have been routed to $pilot::PassPort. By default a data record will go to this port.
Generic
Data records which have been routed to $pilot::FailPort.
The server version when this component was registered
7.5.0.2
Local Component Identifier
2
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allows the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running locally or remotely. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option with care, since it will not increase throughput in all circumstances. Generally speaking, it yields improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
end if;
#parallel;
<p>
A comma-separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (that is, server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be useful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
A comma-separated list of the maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
Parallel Processing Options Server Run On Grid Grid Options
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Use this to specify grid engine specific command line parameters to the job submission command. Leave blank to use defaults.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
<p>A comma-delimited list of global properties, each of which is automatically populated with the name of a file in the job's temporary folder, which always has a unique name.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold only the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Separate property names with commas. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>
<p>Any global property not declared in this way is created on the top-level global property list and is accessible from all subprotocols, but breaks the encapsulation of the subprotocol and may collide with another property of the same name.</p>
<p>Therefore, you should always declare a new global property in the <i>DeclareLocal</i> parameter, to minimize the scope of the global property to the subprotocol where it is used.</p>
<p>The example <i>DeclareLocal</i> value below creates 3 global properties in the subprotocol scope. Note that you can initialize the declared global value with a scalar value using the ":=" syntax, as shown. You can surround text string values with quotes to aid clarity.</p>
<pre>
SubP_Counter := 0, SubP_NameHash, SubP_TitleText:="Sample 1"
</pre>
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Al1HAAAsU3RyaW5nVHlwZQAIAAAAAQEAAQAEAPwAAAAAAAAAAAAQVVJMA8xTz7sBKWh0dHBzOi8vYmxhc3QubmNiaS5ubG0ubmloLmdvdi9CbGFzdC5jZ2kA
StringType
LinkSingleParentParameterType
URL
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
Send to NCBI
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
690 -635
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open
a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display
icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them.
For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols
Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see
<a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel
Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut,
so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating
Shortcuts</a>.
</p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
17.2.0.1267
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Thu Sep 15 15:57:54 2016
Local Component Identifier
4
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@resultFound := 1;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1590 -95
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
5
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The location to which to write the data. Can be a file, a network resource, a global property, or a data record property.
<ul>
<ul>
<li>For a file, enter a path.</li>
<li>For a network resource, enter a network URL in the form <b>protocol://server:port/...</b>. (If port is omitted, a default is used.) Protocols supported are:
<ul>
<li>http, https</li>
<li>ftp</li>
<li>hdfs (Hadoop Distributed File System)</li>
</ul>
</li>
<li>For a global property, enter "global://property-name".</li>
<li>For a data record property, enter "property://property-name" and add an output port.</li>
</ul>
<p>
<b>Note:</b>
</p>
<ul>
<li>When the destination is a file or network resource, be sure to specify an extension such as .txt or .xls. To save in compressed (zipped) format, add the extension .gz or .zip to the filename.</li>
</ul>
Maximum
IfFileExists
Username
Password
Additional Security Info
$(responseXML)
The maximum number of data records to write. If empty, all records are written out.
<p>
If you are writing out data from multiple records to a single binary file, ensure that this is really what you want to do. You may prefer to write out individual record data to separate files in a Run To Completion subprotocol.
Specifies action to take if a file specified by <i>Destination</i> already exists. Options are:
<ul>
<li>Overwrite: Replace the existing file with the new content.
<li>Append: Append the new content to the end of the existing file.
<li>Halt: Raise an error and halt the protocol.
</ul>
Overwrite
Append
Halt
<p>User name for connecting to destination. Primarily used for HDFS.</p>
<p>Password used to connect to destination. Primarily used for HDFS.</p>
<p>This is an array of additional security information. Primarily used by HDFS. </p>
<p>When connecting to secured HDFS the Kerberos KDC Server and realm must be specified. </p>
<b>For example</b>
<p>KDC Server=<Server Name></p>
<p>KDC Realm=<Realm Name></p>
The name of the binary property on the data record to write to file. All other properties are ignored.
responseXML
Open the file in binary mode to output non-textual information
True
False
List of properties to write. (Alias for <i>Binary Data Property</i>.)
$(Binary Data Property)
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
The user who registered this component.
scitegic/generic
The date this component was registered.
Thu Sep 29 10:10:53 2016
Binary Writer
DOCUMENTNODE
{BF2A2E90-418A-4C0B-8550-B2C4D5554628}
1890 -95
The component this component is based upon.
Text Writer
0
100
Writes binary data to a file.
<p>This component writes the content of a binary data property into a named <i>Destination</i> file.
</p>
<p>Ensure that the property specified in <i>Binary Data Property</i> is in fact binary data; it should be a Byte Array value.
</p>
Generic
None
None
The server version when this component was registered
17.2.0.1279
Local Component Identifier
19
The original path within the database of this component.
Components\Data Access and Manipulation\File Writers\Binary Writer
The target URL to send the HTTP request.
HTTP Authentication Options
$(URL)?CMD=Get&FORMAT_TYPE=XML&RID=$(RID)&ALIGNMENT_VIEW=Pairwise&ALIGNMENTS=$(Maximum Hits)&DESCRIPTIONS=100
Options for user credentials when the HTTP server requires authentication.
Authentication Method
Username
Password
<p>
Determines how to obtain the username and password for HTTP authentication.
</p>
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. In this case, the <i>Password</i> parameter is ignored. If left empty, the <i>Username</i> is ignored, but can be filled in to override the user's log-in name.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used if authentication is needed for an HTTP server.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
can be filled in to provide authentication information to the remote server.
</p>
<p>
If the Authentication Method is set to 'Use SciTegic Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username is used along with the user's Pipeline Pilot password.
</p>
The password to be used when authenticating to an HTTP server. This parameter is disabled if the Authentication Method is not 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set Authentication Method to 'Use Username and Password Parameters'");
Parameter('Authentication Method') eq 'Use Username and Password Parameters';
The type of HTTP Method to use when sending the request. These include the standard HTTP Methods of Get, Post, Put, Head, Delete, Trace, Options, and Connect, but can also be an optional extended method, such as the WebDav method "Propfind".
HTTP Body Content
Request Body
Get
<p>When sending parameter data, this parameter specifies the method to use when formatting parameter data into the HTTP request for the remote server, either "FormURLEncoded" or "FormMultipart".</p>
<p>Alternatively, choose the "Send Request Body" option
if you wish to define the body content and content type in the "Request Body" parameter, instead of filling out parameter values. Use this value if you need to send some arbitrary
body content, such as an XML or JSON document. If you select this option, you will need to supply values for the <i>Request Body</i> and for its <i>Content Type</i>.
</p>
SetDisabledDetails("To enable, set HTTP Method to a method that supports an HTTP Body (e.g. Post)");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$';
FormURLEncoded
FormMultipart
Send Request Body
<p>In lieu of specific Http Post parameters, you can also explicitly define the entire HTTP Request body here. Because it is an expression type, specify the name of a property that contains the text of the body.</p>
<p>If this parameter is specified, it overrides the contents of the <i>Parameters</i> group.</p>
Request Body Content Type
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
SetDisabledDetails("To enable, set HTTP Body Content to 'Send Request Body'");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$'
AND
Parameter ('HTTP Body Content') eq 'Send Request Body';
<p>The content-type header value to define the format of the "Request Body" data.</p>
<p>Use a standard MIME type such as the examples below:</p>
<ul>
<li>text/xml;charset=utf-8</li>
<li>application/json</li>
<li>text/plain</li>
</ul>
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
The POST parameters to send to the server. You can add or remove additional parameters by clicking on the ... icon.
Parameter 1
Parameter 2
Parameter 3
Parameter 4
Parameter 5
Parameter 6
SetDisabledDetails("Used for sending a form-structured request body. To enable, set HTTP Method to Post/Put/Patch and set HTTP Body to one of the Form encoding options.");
(Trim(Parameter('HTTP Method')) rlike '^post$|^put$|^patch$')
AND
(Parameter('HTTP Body Content') like 'Form%');
Optionally, you can add additional headers to the HTTP Post request to better control the behavior. On the HTTP request, the data sent in the headers is <b>HeaderName: HeaderValue</b>. For instance, to set HTTP Cookies on the request, set the header name of an additional request header to be <b>Cookie</b> and the value to <b>CookieName=CookieValue; CookieName2=CookieValue2;</b>
Header 1
Each Additional Request Header requires a name and value.
Header 1 Name
Header 1 Value
The name of the Additional HTTP Request Header to send to the remote server.
The value of the Additional HTTP Request Header.
Contains less commonly used parameters
Put Response in Property
Include Response Headers
Additional Allowed Response Codes
Response is Binary
Timeout
Connection Options
Maximum Response Size
The name of the property to store the results from the server. The default is <b>response</b>
responseXML
If set to <b>False</b>, then the only data returned is the HTTP body from the response as indicated by the <i>Put Response in Property</i> parameter. If set to <b>True</b>, then each header on the response will be added as a property to the data record where the name of the property matches the name of the response header. Additionally, a property called <b>HTTP Response</b> contains the first line of the HTTP response headers that includes the HTTP version and HTTP response code.
False
True
<p>A comma separated list of HTTP response codes that are expected from the service. Ordinarily, Http response codes that indicate errors such as "500 Internal Server Error" and "403 Forbidden" are treated as errors by the component and a Pipeline Pilot error is generated. However, errors that can be handled internally by the component such as "401 Unauthorized" and "3xx Redirect" are dealt with. Note that some applications expect response codes and consider response codes to be part of the implementation.</p>
<p>For example, SOAP services often return "500 Internal Server Error" to indicate an error from the service itself that the client might be expecting. For these cases, you can specify a list of response codes such as "500,403" to tell the component to treat those responses as valid response codes.</p>
<p>The response codes that are already handled by the component are:
<ul>
<li>100-199 (processing)</li>
<li>200-299 (valid response)</li>
<li>300-399 (redirect)</li>
<li>401 (HTTP Authentication)</li>
<li>407 (Proxy Authentication)</li>
<li>417 (Expectation failed)</li>
</ul>
<p>For the list of standard HTTP Response codes, see RFC 2616 (<a href="javascript:DoLink('http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html')">http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html</a>). Extended response codes might also be available in other specifications.</p>
<p>Set to <b>True</b> if the response data saved is binary. The <b>Http Connector</b> will enter <i>Streaming Mode</i> where the component, when possible, will attempt to stream the result property directly into a single downstream converter or reader component such as <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components. To use the streamed data within a converter, specify the response property in the downstream component. To use a generic reader, use the format <b>property://ResponsePropertyName</b> within the <b>Source</b> parameter of the reader component. If the streamed property is needed by another component or framework element before reaching the reader/converter component, the result property will be read into memory subject to the maximum size specified by <b>Maximum Response Size</b> and will then be available as a property of type <i>SciTegic.value.ByteArrayValue</i>. This can happen, for instance, if the pipeline splits to more than one component, if a checkpoint is placed on the Http Connector, or if a <a href="javascript:DoLink('{AA4369ED-A1A1-4949-8520-39B23096ACFD}')">Data Record Tree Viewer</a> component is placed between the <b>Http Connector</b> and the downstream reader/converter.</p>
<p><b>Maximum Property Size</b> is not compatible with <i>Streaming Mode</i>, and so is now disabled by default when <b>Response is Binary</b> is true.</p>
<p>For other scenarios where you wish to convert the incoming data based on a specific character set, you can use the Pilotscript function "ConvertBytesToString" to decode the Unicode characters for use in other downstream components.</p>
Detect Character Encodings
Maximum Property Size
True
False
When reading the HTTP response into a text property, international characters are typically encoded in some standard form. The character encoding is often specified as part of the <b>Content-Type</b> header (e.g. Content-Type: text/html; charset=UTF-8). Additionally, some content types such as text/html and text/xml can specify their character encoding within the body text (in an html meta tag or xml doctype tag). Therefore, if this parameter is true and the response will be put into a string property, the component will attempt to determine the character encoding, first from the <b>Content-Type</b> header, then from the body of the message <i>if</i> the Content-Type is specified to be text/html or text/xml.
SetDisabledDetails('Only applicable if Response is Binary is false');
not Parameter('Response is Binary');
True
False
Set the maximum property size. When greater-than zero, the response is sent in one or more new data record properties, until the complete response has been consumed. When empty, the response is sent in one data record property.
SetDisabledDetails('Only applicable if Response is Binary is false and Put Response in Property is set');
not Parameter('Response is Binary') and StrLength(Parameter('Put Response In Property')) > 0;
The time in seconds to wait for a response from the server.
600
<p>Additional HTTP Connection options:</p>
<p><ul>
<li>Allow Http Keep-alive connections (default: <i>true</i>). By default, most HTTP socket connections handle multiple requests to the server. You can disable this if you find that your connection is not handled as expected.</li>
<li>Allow the Http header, Expect: 100 (default: <i>true</i>). When sending a large body to the remote server, Expect 100 allows the remote server to respond mid-stream to inform the client that it needs to authenticate or redirect to a new URL. This can help the Http Connector component avoid sending large bodies to servers multiple times when it turns out that authentication is required or that the URL has been redirected.</li>
<li>Treat Http 403 responses as though a 401 response had been received instead (default: <i>false</i>). When a remote server requires authentication, it is supposed to return a response code of 401 to indicate that credentials are required. However, some servers erroneously send 403 responses instead. Some Http clients deal with this by preemptively sending authentication information. However, this presents a security hole, so this option allows the component developer to selectively send credentials to servers that are responding with 403 Forbidden as though they had actually responded with 401 Authenticate: Basic. It should only be used when it is clear that this will allow integration with a specific remote server.</li>
<li>Force Content-Length Calculation (default: <i>false</i>). This is
only relevant when sending a large body content in the HTTP request. When set, this behavior forces the
component to cache all content before sending it to the server with a content length header. Where the
server supports "chunked" transfer encoding, this setting should be kept off, allowing the
component to start sending "chunks" of the data before knowing the total content length; this
avoids the need to cache all the data before sending it, and so is faster and requires less memory, aiding
scalability.</li>
</ul></p>
Allow Keep-Alive
Allow Expect: 100
Treat 403 Forbidden As 401 Authenticate
Force Content-Length Calculation
The maximum amount of data in bytes to download from the server. The reponse data is truncated once the limit has been reached. If the parameter is blank, the limit is the default value of 64 MB.
64000000
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 1 Name
Parameter 1 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 2 Name
Parameter 2 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 3 Name
Parameter 3 Value
The name of the POST parameter to send to the remote server
RID
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
RID
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 4 Name
Parameter 4 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 5 Name
Parameter 5 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 6 Name
Parameter 6 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
True
False
Halt
Fail
Pass
Halt
Fail
Pass
{8A25024A-C274-485B-BB99-8570DCC4C075}
100
Sends requests to an HTTP server. It can also be used for executing and debugging various HTTP services such as REST and WebDav using methods such as Post, Get, Put, Head, Options.
<p>
Configure this component to make remote requests to an HTTP server, using valid HTTP method verbs like Get, Post and Put. The Source parameter specifies the URL for the location of the HTTP server to receive the HTTP request with the parameter names and values specified by this component. Alternatively, <b>Additional Option</b> parameters may be be configured to execute and consume other types of HTTP services, such as REST and WebDav. The component can either be the start of a new pipeline as a reader component or can be used within an existing pipeline to process multiple requests. When used as a reader component, you can remove the input port to make that role clear.
</p>
<p>
Output from the HTTP connector can be piped into other reader or converter components such as the <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components, to provide inline decoding of the body data. When piping into a reader component, use the format <i>property://ResponsePropertyName</i> within the Source parameter of the reader.
For piping, be sure to set the <b>Additional Options/Response is Binary</b> parameter to <i>True</i> so that character encoding is properly preserved and so that when possible the data can be streamed directly to the downstream component (see the help text for <b>Response is Binary</b> for more details.).
</p>
<p>
Use the <b>HTTP Method</b> parameter to define the method verb of the request. For those methods that include a request body (e.g. Post, Put), the HTTP request may be configured to either send parameter data encoded as form content, or to send a specified body content. Use the <b>HTTP Body</b> parameter to define which of these options is to be used.
</p>
<p>
For a form-encoded request body, fill in the <b>Parameters</b> group. If more than one parameter is needed, you can add additional parameters by pressing the "..." button. Parameter names and values are sent to the remote server using the encoding specified in the <b>HTTP Body</b> parameter.
</p>
<p>
When sending a specified body content (e.g. XML or JSON), set the <b>HTTP Body</b> parameter to indicate that you are using the <b>Request Body</b> parameter, set the actual content in the <b>Request Body</b> parameter and be sure to also define the <b>Content Type</b> using an appropriate MIME type name - and character encoding, if necessary..
</p>
<p>
The response from the server is put into the property named by the <b>Additional Options/Put Response in Property</b> parameter, which is <i>response</i> by default. The handling of the http request and response can be further configured by using the <b>Additional Request Headers</b> and <b>Additional Options</b> groups of parameters. See the individual help of each parameter for further information.</p>
<p><strong>Notes:</strong></p>
<ul>
<li>If the <b>HTTP Method</b> is <i>Post</i>, the request body content must be provided in the <b>Parameters</b> group or in the <b>Request Body</b> parameter. If neither is provided, the component fails.</li>
<li>Cookies are shared between components within a single protocol. To set a custom cookie value, see the PilotScript function <i>HttpSetCookie</i></li>
<li>For remote servers that require HTTP authentication, set the <b>Authentication Options</b> parameters to provide username and password information either through parameters or through single sign-on from the Pipeline Pilot server to the remote server.</li>
<li>For performing basic HTTP Get requests that do not require any special headers or options, the majority of the standard file reader components such as the <a href="javascript:DoLink('{8056E8AD-33EA-42BB-9099-EAB24C3D08E6}')">Text Reader</a> and <a href="javascript:DoLink('{B0B1B56C-3DF6-444C-BB78-B43211188F63}')">Binary Reader</a> can read URLs as sources directly and the <a href="javascript:DoLink('{38A20EDC-AAAA-42EF-97BA-D1B40B74FFE0}')">Copy File</a> component can be used if HTTP authentication is required by the server.</li>
<li>When piping the output of the HTTP Connector into other readers, always set the <b>Additional Options - Response is Binary</b> parameter to <i>true</i>. This ensures that character encoding is properly preserved and that streaming mode can be used when possible.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
9.5.0.666
Local Component Identifier
7
1290 -275
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Dec 08 10:44:31 2014
HTTP Connector
DOCUMENTNODE
0
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Web Service Integration\HTTP Connector
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
if @SeqCount==1 and @'Result Text Property' ne '' then
@resultText := convertbytestostring(responseXML);
end if;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Save Result Text
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1590 -455
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
8
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
end if;
#parallel;
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
Parallel Processing Options Server Run On Grid Grid Options
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Use this to specify grid engine specific command line parameters to the job submission command. Leave blank to use defaults.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
<p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.</p><p>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Get response
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
690 -1175
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them. For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut, so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating Shortcuts</a>.
</p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
9.5.0.623
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Wed Oct 29 10:13:55 2014
Local Component Identifier
10
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
The target URL to send the HTTP request.
HTTP Authentication Options
$(URL)?CMD=Get&FORMAT_OBJECT=SearchInfo&RID=$(RID)
Options for user credentials when the HTTP server requires authentication.
Authentication Method
Username
Password
<p>
Determines how to obtain the username and password for HTTP authentication.
</p>
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. In this case, the <i>Password</i> parameter is ignored. If left empty, the <i>Username</i> is ignored, but can be filled in to override the user's log-in name.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used if authentication is needed for an HTTP server.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
can be filled in to provide authentication information to the remote server.
</p>
<p>
If the Authentication Method is set to 'Use SciTegic Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username is used along with the user's Pipeline Pilot password.
</p>
The password to be used when authenticating to an HTTP server. This parameter is disabled if the Authentication Method is not 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set Authentication Method to 'Use Username and Password Parameters'");
Parameter('Authentication Method') eq 'Use Username and Password Parameters';
The type of HTTP Method to use when sending the request. These include the standard HTTP Methods of Get, Post, Put, Head, Delete, Trace, Options, and Connect, but can also be an optional extended method, such as the WebDav method "Propfind".
HTTP Body Content
Request Body
Get
<p>When sending parameter data, this parameter specifies the method to use when formatting parameter data into the HTTP request for the remote server, either "FormURLEncoded" or "FormMultipart".</p>
<p>Alternatively, choose the "Send Request Body" option
if you wish to define the body content and content type in the "Request Body" parameter, instead of filling out parameter values. Use this value if you need to send some arbitrary
body content, such as an XML or JSON document. If you select this option, you will need to supply values for the <i>Request Body</i> and for its <i>Content Type</i>.
</p>
SetDisabledDetails("To enable, set HTTP Method to a method that supports an HTTP Body (e.g. Post)");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$';
FormURLEncoded
FormMultipart
Send Request Body
<p>In lieu of specific Http Post parameters, you can also explicitly define the entire HTTP Request body here. Because it is an expression type, specify the name of a property that contains the text of the body.</p>
<p>If this parameter is specified, it overrides the contents of the <i>Parameters</i> group.</p>
Request Body Content Type
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
SetDisabledDetails("To enable, set HTTP Body Content to 'Send Request Body'");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$'
AND
Parameter ('HTTP Body Content') eq 'Send Request Body';
<p>The content-type header value to define the format of the "Request Body" data.</p>
<p>Use a standard MIME type such as the examples below:</p>
<ul>
<li>text/xml;charset=utf-8</li>
<li>application/json</li>
<li>text/plain</li>
</ul>
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
The POST parameters to send to the server. You can add or remove additional parameters by clicking on the ... icon.
Parameter 1
SetDisabledDetails("Used for sending a form-structured request body. To enable, set HTTP Method to Post/Put/Patch and set HTTP Body to one of the Form encoding options.");
(Trim(Parameter('HTTP Method')) rlike '^post$|^put$|^patch$')
AND
(Parameter('HTTP Body Content') like 'Form%');
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 1 Name
Parameter 1 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
Optionally, you can add additional headers to the HTTP Post request to better control the behavior. On the HTTP request, the data sent in the headers is <b>HeaderName: HeaderValue</b>. For instance, to set HTTP Cookies on the request, set the header name of an additional request header to be <b>Cookie</b> and the value to <b>CookieName=CookieValue; CookieName2=CookieValue2;</b>
Header 1
Each Additional Request Header requires a name and value.
Header 1 Name
Header 1 Value
The name of the Additional HTTP Request Header to send to the remote server.
The value of the Additional HTTP Request Header.
Contains less commonly used parameters
Put Response in Property
Include Response Headers
Additional Allowed Response Codes
Response is Binary
Timeout
Connection Options
Maximum Response Size
The name of the property to store the results from the server. The default is <b>response</b>
responseStatus
If set to <b>False</b>, then the only data returned is the HTTP body from the response as indicated by the <i>Put Response in Property</i> parameter. If set to <b>True</b>, then each header on the response will be added as a property to the data record where the name of the property matches the name of the response header. Additionally, a property called <b>HTTP Response</b> contains the first line of the HTTP response headers that includes the HTTP version and HTTP response code.
False
True
<p>A comma separated list of HTTP response codes that are expected from the service. Ordinarily, Http response codes that indicate errors such as "500 Internal Server Error" and "403 Forbidden" are treated as errors by the component and a Pipeline Pilot error is generated. However, errors that can be handled internally by the component such as "401 Unauthorized" and "3xx Redirect" are dealt with. Note that some applications expect response codes and consider response codes to be part of the implementation.</p>
<p>For example, SOAP services often return "500 Internal Server Error" to indicate an error from the service itself that the client might be expecting. For these cases, you can specify a list of response codes such as "500,403" to tell the component to treat those responses as valid response codes.</p>
<p>The response codes that are already handled by the component are:
<ul>
<li>100-199 (processing)</li>
<li>200-299 (valid response)</li>
<li>300-399 (redirect)</li>
<li>401 (HTTP Authentication)</li>
<li>407 (Proxy Authentication)</li>
<li>417 (Expectation failed)</li>
</ul>
<p>For the list of standard HTTP Response codes, see RFC 2616 (<a href="javascript:DoLink('http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html')">http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html</a>). Extended response codes might also be available in other specifications.</p>
<p>Set to <b>True</b> if the response data saved is binary. The <b>Http Connector</b> will enter <i>Streaming Mode</i> where the component, when possible, will attempt to stream the result property directly into a single downstream converter or reader component such as <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components. To use the streamed data within a converter, specify the response property in the downstream component. To use a generic reader, use the format <b>property://ResponsePropertyName</b> within the <b>Source</b> parameter of the reader component. If the streamed property is needed by another component or framework element before reaching the reader/converter component, the result property will be read into memory subject to the maximum size specified by <b>Maximum Response Size</b> and will then be available as a property of type <i>SciTegic.value.ByteArrayValue</i>. This can happen, for instance, if the pipeline splits to more than one component, if a checkpoint is placed on the Http Connector, or if a <a href="javascript:DoLink('{AA4369ED-A1A1-4949-8520-39B23096ACFD}')">Data Record Tree Viewer</a> component is placed between the <b>Http Connector</b> and the downstream reader/converter.</p>
<p><b>Maximum Property Size</b> is not compatible with <i>Streaming Mode</i>, and so is now disabled by default when <b>Response is Binary</b> is true.</p>
<p>For other scenarios where you wish to convert the incoming data based on a specific character set, you can use the Pilotscript function "ConvertBytesToString" to decode the Unicode characters for use in other downstream components.</p>
Detect Character Encodings
Maximum Property Size
True
False
When reading the HTTP response into a text property, international characters are typically encoded in some standard form. The character encoding is often specified as part of the <b>Content-Type</b> header (e.g. Content-Type: text/html; charset=UTF-8). Additionally, some content types such as text/html and text/xml can specify their character encoding within the body text (in an html meta tag or xml doctype tag). Therefore, if this parameter is true and the response will be put into a string property, the component will attempt to determine the character encoding, first from the <b>Content-Type</b> header, then from the body of the message <i>if</i> the Content-Type is specified to be text/html or text/xml.
SetDisabledDetails('Only applicable if Response is Binary is false');
not Parameter('Response is Binary');
True
False
Set the maximum property size. When greater-than zero, the response is sent in one or more new data record properties, until the complete response has been consumed. When empty, the response is sent in one data record property.
SetDisabledDetails('Only applicable if Response is Binary is false and Put Response in Property is set');
not Parameter('Response is Binary') and StrLength(Parameter('Put Response In Property')) > 0;
The time in seconds to wait for a response from the server.
600
<p>Additional HTTP Connection options:</p>
<p><ul>
<li>Allow Http Keep-alive connections (default: <i>true</i>). By default, most HTTP socket connections handle multiple requests to the server. You can disable this if you find that your connection is not handled as expected.</li>
<li>Allow the Http header, Expect: 100 (default: <i>true</i>). When sending a large body to the remote server, Expect 100 allows the remote server to respond mid-stream to inform the client that it needs to authenticate or redirect to a new URL. This can help the Http Connector component avoid sending large bodies to servers multiple times when it turns out that authentication is required or that the URL has been redirected.</li>
<li>Treat Http 403 responses as though a 401 response had been received instead (default: <i>false</i>). When a remote server requires authentication, it is supposed to return a response code of 401 to indicate that credentials are required. However, some servers erroneously send 403 responses instead. Some Http clients deal with this by preemptively sending authentication information. However, this presents a security hole, so this option allows the component developer to selectively send credentials to servers that are responding with 403 Forbidden as though they had actually responded with 401 Authenticate: Basic. It should only be used when it is clear that this will allow integration with a specific remote server.</li>
<li>Force Content-Length Calculation (default: <i>false</i>). This is
only relevant when sending a large body content in the HTTP request. When set, this behavior forces the
component to cache all content before sending it to the server with a content length header. Where the
server supports "chunked" transfer encoding, this setting should be kept off, allowing the
component to start sending "chunks" of the data before knowing the total content length; this
avoids the need to cache all the data before sending it, and so is faster and requires less memory, aiding
scalability.</li>
</ul></p>
Allow Keep-Alive
Allow Expect: 100
Treat 403 Forbidden As 401 Authenticate
Force Content-Length Calculation
The maximum amount of data in bytes to download from the server. The reponse data is truncated once the limit has been reached. If the parameter is blank, the limit is the default value of 64 MB.
64000000
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
True
False
Halt
Fail
Pass
Halt
Fail
Pass
{8A25024A-C274-485B-BB99-8570DCC4C075}
100
Sends requests to an HTTP server. It can also be used for executing and debugging various HTTP services such as REST and WebDav using methods such as Post, Get, Put, Head, Options.
<p>
Configure this component to make remote requests to an HTTP server, using valid HTTP method verbs like Get, Post and Put. The Source parameter specifies the URL for the location of the HTTP server to receive the HTTP request with the parameter names and values specified by this component. Alternatively, <b>Additional Option</b> parameters may be be configured to execute and consume other types of HTTP services, such as REST and WebDav. The component can either be the start of a new pipeline as a reader component or can be used within an existing pipeline to process multiple requests. When used as a reader component, you can remove the input port to make that role clear.
</p>
<p>
Output from the HTTP connector can be piped into other reader or converter components such as the <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components, to provide inline decoding of the body data. When piping into a reader component, use the format <i>property://ResponsePropertyName</i> within the Source parameter of the reader.
For piping, be sure to set the <b>Additional Options/Response is Binary</b> parameter to <i>True</i> so that character encoding is properly preserved and so that when possible the data can be streamed directly to the downstream component (see the help text for <b>Response is Binary</b> for more details.).
</p>
<p>
Use the <b>HTTP Method</b> parameter to define the method verb of the request. For those methods that include a request body (e.g. Post, Put), the HTTP request may be configured to either send parameter data encoded as form content, or to send a specified body content. Use the <b>HTTP Body</b> parameter to define which of these options is to be used.
</p>
<p>
For a form-encoded request body, fill in the <b>Parameters</b> group. If more than one parameter is needed, you can add additional parameters by pressing the "..." button. Parameter names and values are sent to the remote server using the encoding specified in the <b>HTTP Body</b> parameter.
</p>
<p>
When sending a specified body content (e.g. XML or JSON), set the <b>HTTP Body</b> parameter to indicate that you are using the <b>Request Body</b> parameter, set the actual content in the <b>Request Body</b> parameter and be sure to also define the <b>Content Type</b> using an appropriate MIME type name - and character encoding, if necessary..
</p>
<p>
The response from the server is put into the property named by the <b>Additional Options/Put Response in Property</b> parameter, which is <i>response</i> by default. The handling of the http request and response can be further configured by using the <b>Additional Request Headers</b> and <b>Additional Options</b> groups of parameters. See the individual help of each parameter for further information.</p>
<p><strong>Notes:</strong></p>
<ul>
<li>If the <b>HTTP Method</b> is <i>Post</i>, the request body content must be provided in the <b>Parameters</b> group or in the <b>Request Body</b> parameter. If neither is provided, the component fails.</li>
<li>Cookies are shared between components within a single protocol. To set a custom cookie value, see the PilotScript function <i>HttpSetCookie</i></li>
<li>For remote servers that require HTTP authentication, set the <b>Authentication Options</b> parameters to provide username and password information either through parameters or through single sign-on from the Pipeline Pilot server to the remote server.</li>
<li>For performing basic HTTP Get requests that do not require any special headers or options, the majority of the standard file reader components such as the <a href="javascript:DoLink('{8056E8AD-33EA-42BB-9099-EAB24C3D08E6}')">Text Reader</a> and <a href="javascript:DoLink('{B0B1B56C-3DF6-444C-BB78-B43211188F63}')">Binary Reader</a> can read URLs as sources directly and the <a href="javascript:DoLink('{38A20EDC-AAAA-42EF-97BA-D1B40B74FFE0}')">Copy File</a> component can be used if HTTP authentication is required by the server.</li>
<li>When piping the output of the HTTP Connector into other readers, always set the <b>Additional Options - Response is Binary</b> parameter to <i>true</i>. This ensures that character encoding is properly preserved and that streaming mode can be used when possible.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
9.5.0.666
Local Component Identifier
0
90 -95
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Dec 08 10:44:31 2014
HTTP Connector
DOCUMENTNODE
0
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Web Service Integration\HTTP Connector
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
use strict;
use warnings;
use Bio::Tools::Run::RemoteBlast;
use SciTegic::Bio::PipelinePilot::FromBioPerl::AddSearchResults;
use SciTegic::Bio::Tools::Blast::BlastUtilities;
use SciTegic::Bio::PipelinePilot::ToBioPerl::CreateBioPerlRichSequence;
use SciTegic::Bio::PipelinePilot::Utils::PipelinePilotUtilities;
use SciTegic::Bio::PipelinePilot::Validate::SequenceValidation;
sub onInitialize
{
my $context = shift;
return pilot::READYFORINPUTDATA;
}
sub onProcess
{
my $context = shift;
my $data = shift;
# make sure that pipeline statistics on the client get updated
# (currently needed for computationally intensive tasks)
$context->sendMessageToClient("Running remote BLAST");
my $propertiesHashRef = $data->getProperties->getHashRef();
my $response = $propertiesHashRef->{'responseStatus'};
if( length($response)>0 ) {
my $status;
if($response =~ /Status=(WAITING|ERROR|FAILED|UNKNOWN|READY)/i ) {
$status = $1;
pilot::debugMessage("status is $status");
if( $status eq 'ERROR' ) {
die('Online BLAST server error');
} elsif( $status eq 'FAILED' ) {
die('Online BLAST request failed');
}
} else {
die('Online BLAST failed. No status returned.');
}
if ( $status ne 'READY' ) {
$data->routeTo(pilot::FAILPORT);
}
$context->sendMessageToClient(""); # we're done, so clear the status message
return pilot::READYFORINPUTDATA;
}
}
sub onFinalize
{
my $context = shift;
}
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/seqanal
<p>
The Perl component functions by creating an instance of a Perl interpreter and
passing the supplied Perl script to the interpreter for execution. The
creation of the Perl interpreter occurs one time, while script execution
occurs once for initialization and finalization subroutines and once
for each data record processed by the component.
</p>
<p>
However, in the context of a Run To Completion (RTC) subprtocols, the Perl
component would normally be re-instantiated for each data record,
in common with other components in an RTC subprotocol. This implies the
re-creation of the Perl interpreter for each data record, which can be
relatively time consuming, particularly for Perl scripts which
themselves perform a rapid operation per data record.
</p>
<p>
So, to avoid significant differences in performance timings between RTC
and non-RTC contexts, the Perl component's default behavior is to <i>keep</i>
the same instance of the Perl interpreter throughout its life cycle. This
is indicated by the False setting of the <em>Rebuild In RTC</em> parameter.
</p>
<p>
However, there are some circumstances where Perl script or 3rd party modules
are not written in a way that allows them to safely re-use the same interpreter in this way. So if you see unexpected behavior from a Perl
component in a Run To Completion subprotocol, switch the <em>Rebuild In RTC</em>
parameter to True to see if that resolves the issue.
</p>
<p>
If that fixes the problem, and the Perl cannot be re-cast to
handle the persistent interpreter,
you may need to trade performance for stability.
</p>
True
False
The version of Perl to run against. Version 5.8.9 is the default. Version 5.16.2 is only available on 64-bit server builds.
5.8.9
5.16.2
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Tue Feb 05 15:52:10 2013
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Check if ready
990 -1175
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
<li> If you see unexpected or erroneous behavior when using this component
in a Run To Completion subprotocol, please look at the help text for the
<em>Rebuild In RTC</em> parameter, which is in the Implementation tab.</li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_comp_dev.pdf')">Perl Component Development Guide</a></li>
</ul>
Generic
Generic
Generic
The server version when this component was registered
9.0.1.15
Local Component Identifier
11
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
Al1HAAEpVGhlIHRhcmdldCBVUkwgdG8gc2VuZCB0aGUgSFRUUCByZXF1ZXN0LiRUZXh0VHlwZQAMAAAAAAEAAQQEAfwAAAAAAAAAAAAcU291cmNlA8xTz7sBKWh0dHBzOi8vYmxhc3QubmNiaS5ubG0ubmloLmdvdi9CbGFzdC5jZ2kA
TextType
LinkSingleParentParameterType
The target URL to send the HTTP request.
HTTP Authentication Options
URL
Options for user credentials when the HTTP server requires authentication.
Authentication Method
Username
Password
<p>
Determines how to obtain the username and password for HTTP authentication.
</p>
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. In this case, the <i>Password</i> parameter is ignored. If left empty, the <i>Username</i> is ignored, but can be filled in to override the user's log-in name.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used if authentication is needed for an HTTP server.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
can be filled in to provide authentication information to the remote server.
</p>
<p>
If the Authentication Method is set to 'Use SciTegic Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username is used along with the user's Pipeline Pilot password.
</p>
The password to be used when authenticating to an HTTP server. This parameter is disabled if the Authentication Method is not 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set Authentication Method to 'Use Username and Password Parameters'");
Parameter('Authentication Method') eq 'Use Username and Password Parameters';
The type of HTTP Method to use when sending the request. These include the standard HTTP Methods of Get, Post, Put, Head, Delete, Trace, Options, and Connect, but can also be an optional extended method, such as the WebDav method "Propfind".
HTTP Body Content
Request Body
Post
<p>When sending parameter data, this parameter specifies the method to use when formatting parameter data into the HTTP request for the remote server, either "FormURLEncoded" or "FormMultipart".</p>
<p>Alternatively, choose the "Send Request Body" option
if you wish to define the body content and content type in the "Request Body" parameter, instead of filling out parameter values. Use this value if you need to send some arbitrary
body content, such as an XML or JSON document. If you select this option, you will need to supply values for the <i>Request Body</i> and for its <i>Content Type</i>.
</p>
SetDisabledDetails("To enable, set HTTP Method to a method that supports an HTTP Body (e.g. Post)");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$';
FormURLEncoded
FormMultipart
Send Request Body
<p>In lieu of specific Http Post parameters, you can also explicitly define the entire HTTP Request body here. Because it is an expression type, specify the name of a property that contains the text of the body.</p>
<p>If this parameter is specified, it overrides the contents of the <i>Parameters</i> group.</p>
Request Body Content Type
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
SetDisabledDetails("To enable, set HTTP Body Content to 'Send Request Body'");
Trim(Parameter('HTTP Method')) not rlike '^Get$|^Head$|^Options$'
AND
Parameter ('HTTP Body Content') eq 'Send Request Body';
requestBody
<p>The content-type header value to define the format of the "Request Body" data.</p>
<p>Use a standard MIME type such as the examples below:</p>
<ul>
<li>text/xml;charset=utf-8</li>
<li>application/json</li>
<li>text/plain</li>
</ul>
SetInvalidDetails(
ParameterName() . ' must be filled in when sending the raw request body.');
Stringlen(Parameter()) > 0;
application/x-www-form-urlencoded
The POST parameters to send to the server. You can add or remove additional parameters by clicking on the ... icon.
Parameter 1
SetDisabledDetails("Used for sending a form-structured request body. To enable, set HTTP Method to Post/Put/Patch and set HTTP Body to one of the Form encoding options.");
(Trim(Parameter('HTTP Method')) rlike '^post$|^put$|^patch$')
AND
(Parameter('HTTP Body Content') like 'Form%');
<p>Each POST parameter requires a name and value.</p>
<p>You can also define metadata values for each parameter
which can be useful in the multipart/form context.
<ul>
<li>To associate a content type with a parameter, set the parameter
metadata value "content-type" to the appropriate mime type.</li>
<li>To associate a filename with the parameter, define a metadata value named "filename" and set it to the value that you wish to define for the parameter content.
</li>
</ul>
Parameter 1 Name
Parameter 1 Value
The name of the POST parameter to send to the remote server
The value of the POST parameter. This can be any Pilotscript expression that returns a string. To specify a literal value, enclose it within quotation marks (for example, 'myvalue'). To specify a property name, type it in (for example, myproperty)
Optionally, you can add additional headers to the HTTP Post request to better control the behavior. On the HTTP request, the data sent in the headers is <b>HeaderName: HeaderValue</b>. For instance, to set HTTP Cookies on the request, set the header name of an additional request header to be <b>Cookie</b> and the value to <b>CookieName=CookieValue; CookieName2=CookieValue2;</b>
Header 1
Each Additional Request Header requires a name and value.
Header 1 Name
Header 1 Value
The name of the Additional HTTP Request Header to send to the remote server.
The value of the Additional HTTP Request Header.
Contains less commonly used parameters
Put Response in Property
Include Response Headers
Additional Allowed Response Codes
Response is Binary
Timeout
Connection Options
Maximum Response Size
The name of the property to store the results from the server. The default is <b>response</b>
response
If set to <b>False</b>, then the only data returned is the HTTP body from the response as indicated by the <i>Put Response in Property</i> parameter. If set to <b>True</b>, then each header on the response will be added as a property to the data record where the name of the property matches the name of the response header. Additionally, a property called <b>HTTP Response</b> contains the first line of the HTTP response headers that includes the HTTP version and HTTP response code.
False
True
<p>A comma separated list of HTTP response codes that are expected from the service. Ordinarily, Http response codes that indicate errors such as "500 Internal Server Error" and "403 Forbidden" are treated as errors by the component and a Pipeline Pilot error is generated. However, errors that can be handled internally by the component such as "401 Unauthorized" and "3xx Redirect" are dealt with. Note that some applications expect response codes and consider response codes to be part of the implementation.</p>
<p>For example, SOAP services often return "500 Internal Server Error" to indicate an error from the service itself that the client might be expecting. For these cases, you can specify a list of response codes such as "500,403" to tell the component to treat those responses as valid response codes.</p>
<p>The response codes that are already handled by the component are:
<ul>
<li>100-199 (processing)</li>
<li>200-299 (valid response)</li>
<li>300-399 (redirect)</li>
<li>401 (HTTP Authentication)</li>
<li>407 (Proxy Authentication)</li>
<li>417 (Expectation failed)</li>
</ul>
<p>For the list of standard HTTP Response codes, see RFC 2616 (<a href="javascript:DoLink('http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html')">http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html</a>). Extended response codes might also be available in other specifications.</p>
<p>Set to <b>True</b> if the response data saved is binary. The <b>Http Connector</b> will enter <i>Streaming Mode</i> where the component, when possible, will attempt to stream the result property directly into a single downstream converter or reader component such as <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components. To use the streamed data within a converter, specify the response property in the downstream component. To use a generic reader, use the format <b>property://ResponsePropertyName</b> within the <b>Source</b> parameter of the reader component. If the streamed property is needed by another component or framework element before reaching the reader/converter component, the result property will be read into memory subject to the maximum size specified by <b>Maximum Response Size</b> and will then be available as a property of type <i>SciTegic.value.ByteArrayValue</i>. This can happen, for instance, if the pipeline splits to more than one component, if a checkpoint is placed on the Http Connector, or if a <a href="javascript:DoLink('{AA4369ED-A1A1-4949-8520-39B23096ACFD}')">Data Record Tree Viewer</a> component is placed between the <b>Http Connector</b> and the downstream reader/converter.</p>
<p><b>Maximum Property Size</b> is not compatible with <i>Streaming Mode</i>, and so is now disabled by default when <b>Response is Binary</b> is true.</p>
<p>For other scenarios where you wish to convert the incoming data based on a specific character set, you can use the Pilotscript function "ConvertBytesToString" to decode the Unicode characters for use in other downstream components.</p>
Detect Character Encodings
Maximum Property Size
True
False
When reading the HTTP response into a text property, international characters are typically encoded in some standard form. The character encoding is often specified as part of the <b>Content-Type</b> header (e.g. Content-Type: text/html; charset=UTF-8). Additionally, some content types such as text/html and text/xml can specify their character encoding within the body text (in an html meta tag or xml doctype tag). Therefore, if this parameter is true and the response will be put into a string property, the component will attempt to determine the character encoding, first from the <b>Content-Type</b> header, then from the body of the message <i>if</i> the Content-Type is specified to be text/html or text/xml.
SetDisabledDetails('Only applicable if Response is Binary is false');
not Parameter('Response is Binary');
True
False
Set the maximum property size. When greater-than zero, the response is sent in one or more new data record properties, until the complete response has been consumed. When empty, the response is sent in one data record property.
SetDisabledDetails('Only applicable if Response is Binary is false and Put Response in Property is set');
not Parameter('Response is Binary') and StrLength(Parameter('Put Response In Property')) > 0;
The time in seconds to wait for a response from the server.
30
<p>Additional HTTP Connection options:</p>
<p><ul>
<li>Allow Http Keep-alive connections (default: <i>true</i>). By default, most HTTP socket connections handle multiple requests to the server. You can disable this if you find that your connection is not handled as expected.</li>
<li>Allow the Http header, Expect: 100 (default: <i>true</i>). When sending a large body to the remote server, Expect 100 allows the remote server to respond mid-stream to inform the client that it needs to authenticate or redirect to a new URL. This can help the Http Connector component avoid sending large bodies to servers multiple times when it turns out that authentication is required or that the URL has been redirected.</li>
<li>Treat Http 403 responses as though a 401 response had been received instead (default: <i>false</i>). When a remote server requires authentication, it is supposed to return a response code of 401 to indicate that credentials are required. However, some servers erroneously send 403 responses instead. Some Http clients deal with this by preemptively sending authentication information. However, this presents a security hole, so this option allows the component developer to selectively send credentials to servers that are responding with 403 Forbidden as though they had actually responded with 401 Authenticate: Basic. It should only be used when it is clear that this will allow integration with a specific remote server.</li>
<li>Force Content-Length Calculation (default: <i>false</i>). This is
only relevant when sending a large body content in the HTTP request. When set, this behavior forces the
component to cache all content before sending it to the server with a content length header. Where the
server supports "chunked" transfer encoding, this setting should be kept off, allowing the
component to start sending "chunks" of the data before knowing the total content length; this
avoids the need to cache all the data before sending it, and so is faster and requires less memory, aiding
scalability.</li>
</ul></p>
Allow Keep-Alive
Allow Expect: 100
Treat 403 Forbidden As 401 Authenticate
Force Content-Length Calculation
The maximum amount of data in bytes to download from the server. The reponse data is truncated once the limit has been reached. If the parameter is blank, the limit is the default value of 64 MB.
64000000
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
{8A25024A-C274-485B-BB99-8570DCC4C075}
100
Sends requests to an HTTP server. It can also be used for executing and debugging various HTTP services such as REST and WebDav using methods such as Post, Get, Put, Head, Options.
<p>
Configure this component to make remote requests to an HTTP server, using valid HTTP method verbs like Get, Post and Put. The Source parameter specifies the URL for the location of the HTTP server to receive the HTTP request with the parameter names and values specified by this component. Alternatively, <b>Additional Option</b> parameters may be be configured to execute and consume other types of HTTP services, such as REST and WebDav. The component can either be the start of a new pipeline as a reader component or can be used within an existing pipeline to process multiple requests. When used as a reader component, you can remove the input port to make that role clear.
</p>
<p>
Output from the HTTP connector can be piped into other reader or converter components such as the <a href="javascript:DoLink('{93C4D73F-0B4D-03F4-B0E3-D82C4FE04C6F}')">XML Reader (Generic)</a>, <a href="javascript:DoLink('{40A5F3DC-E6D3-4781-BB00-D91254C5350E}')">JSON Reader (Generic)</a>, <a href="javascript:DoLink('{CE3E731B-BA0B-4EEE-9532-9E9B66D1F6DF}')">Data from XML</a>, or <a href="javascript:DoLink('{C515C623-DA9A-414E-BD66-C14A922EFA60}')">Data from JSON</a> components, to provide inline decoding of the body data. When piping into a reader component, use the format <i>property://ResponsePropertyName</i> within the Source parameter of the reader.
For piping, be sure to set the <b>Additional Options/Response is Binary</b> parameter to <i>True</i> so that character encoding is properly preserved and so that when possible the data can be streamed directly to the downstream component (see the help text for <b>Response is Binary</b> for more details.).
</p>
<p>
Use the <b>HTTP Method</b> parameter to define the method verb of the request. For those methods that include a request body (e.g. Post, Put), the HTTP request may be configured to either send parameter data encoded as form content, or to send a specified body content. Use the <b>HTTP Body</b> parameter to define which of these options is to be used.
</p>
<p>
For a form-encoded request body, fill in the <b>Parameters</b> group. If more than one parameter is needed, you can add additional parameters by pressing the "..." button. Parameter names and values are sent to the remote server using the encoding specified in the <b>HTTP Body</b> parameter.
</p>
<p>
When sending a specified body content (e.g. XML or JSON), set the <b>HTTP Body</b> parameter to indicate that you are using the <b>Request Body</b> parameter, set the actual content in the <b>Request Body</b> parameter and be sure to also define the <b>Content Type</b> using an appropriate MIME type name - and character encoding, if necessary..
</p>
<p>
The response from the server is put into the property named by the <b>Additional Options/Put Response in Property</b> parameter, which is <i>response</i> by default. The handling of the http request and response can be further configured by using the <b>Additional Request Headers</b> and <b>Additional Options</b> groups of parameters. See the individual help of each parameter for further information.</p>
<p><strong>Notes:</strong></p>
<ul>
<li>If the <b>HTTP Method</b> is <i>Post</i>, the request body content must be provided in the <b>Parameters</b> group or in the <b>Request Body</b> parameter. If neither is provided, the component fails.</li>
<li>Cookies are shared between components within a single protocol. To set a custom cookie value, see the PilotScript function <i>HttpSetCookie</i></li>
<li>For remote servers that require HTTP authentication, set the <b>Authentication Options</b> parameters to provide username and password information either through parameters or through single sign-on from the Pipeline Pilot server to the remote server.</li>
<li>For performing basic HTTP Get requests that do not require any special headers or options, the majority of the standard file reader components such as the <a href="javascript:DoLink('{8056E8AD-33EA-42BB-9099-EAB24C3D08E6}')">Text Reader</a> and <a href="javascript:DoLink('{B0B1B56C-3DF6-444C-BB78-B43211188F63}')">Binary Reader</a> can read URLs as sources directly and the <a href="javascript:DoLink('{38A20EDC-AAAA-42EF-97BA-D1B40B74FFE0}')">Copy File</a> component can be used if HTTP authentication is required by the server.</li>
<li>When piping the output of the HTTP Connector into other readers, always set the <b>Additional Options - Response is Binary</b> parameter to <i>true</i>. This ensures that character encoding is properly preserved and that streaming mode can be used when possible.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
9.5.0.666
Local Component Identifier
0
90 -1355
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Dec 08 10:44:31 2014
HTTP Connector
DOCUMENTNODE
0
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Web Service Integration\HTTP Connector
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
package main;
#
# RemoteBlastGetRID.pm
#
# This program and all files, classes and data used by it
# are protected by Copyright and hence may not be used, copied,
# modified, transmitted, inspected, or executed by any means including
# the use of electronic data processing equipment, xerography, or
# any other methods without the express written permission of the
# copyright holder.
#
# Copyright (C) 2017 Accelrys Software Inc., All Rights Reserved
#
use strict;
use warnings;
use Bio::Tools::Run::RemoteBlast;
use SciTegic::Bio::PipelinePilot::FromBioPerl::AddSearchResults;
use SciTegic::Bio::Tools::Blast::BlastUtilities;
use SciTegic::Bio::PipelinePilot::ToBioPerl::CreateBioPerlRichSequence;
use SciTegic::Bio::PipelinePilot::Utils::PipelinePilotUtilities;
use SciTegic::Bio::PipelinePilot::Validate::SequenceValidation;
sub onInitialize
{
my $context = shift;
return pilot::READYFORINPUTDATA;
}
sub onProcess
{
my $context = shift;
my $data = shift;
# make sure that pipeline statistics on the client get updated
# (currently needed for computationally intensive tasks)
$context->sendMessageToClient("Running remote BLAST");
my $globalsHashRef = $context->getGlobalProperties->getHashRef();
my $propertiesHashRef = $data->getProperties->getHashRef();
my $response = $propertiesHashRef->{'response'};
my $resultCode = getBlastResults($context, $response, $propertiesHashRef, $globalsHashRef);
if ($resultCode <= 0){
$data->routeTo(pilot::FAILPORT);
}
$context->sendMessageToClient(""); # we're done, so clear the status message
return pilot::READYFORINPUTDATA;
}
sub onFinalize
{
my $context = shift;
}
sub getBlastResults
{
my ($context, $response, $propertiesHashRef, $globalsHashRef) = @_;
my $ridFound = 0;
my $rtoe = 0;
eval
{
# need to not stomp Pipeline Pilot's die handler
local $SIG{__DIE__};
if( length($response)>0 ){
my @subdata = split(/\n/, $response );
my $inQBlastInfoBegin = 0;
foreach ( @subdata ) {
# example:
#<!--QBlastInfoBegin
# RID = 5ZU8ZZFN016
# RTOE = 32
#QBlastInfoEnd
last if (/QBlastInfoEnd/);
$inQBlastInfoBegin = 1 if (/QBlastInfoBegin/);
if($inQBlastInfoBegin) {
if(/^\s*RID\s*=\s*(\S+)/) {
$propertiesHashRef->{'RID'} = $1;
$globalsHashRef->{'RID'} = $1;
pilot::debugMessage("rid is $1");
$ridFound=1;
}
if(/^\s*RTOE\s*=\s*(\d+)/) {
$rtoe=$1;
pilot::debugMessage("RTOE (estimated time in seconds until the search is completed) is $rtoe");
}
}
}
}
};
if( $ridFound == 0 ) {
$propertiesHashRef->{'blastError'} = 'No RID found. BLAST search failed.';
}
if ($@)
{
die("Remote BLAST failed: $@\n");
}
sleep($rtoe) if($rtoe>0);
return $ridFound;
}
1;
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/seqanal
<p>
The Perl component functions by creating an instance of a Perl interpreter and
passing the supplied Perl script to the interpreter for execution. The
creation of the Perl interpreter occurs one time, while script execution
occurs once for initialization and finalization subroutines and once
for each data record processed by the component.
</p>
<p>
However, in the context of a Run To Completion (RTC) subprtocols, the Perl
component would normally be re-instantiated for each data record,
in common with other components in an RTC subprotocol. This implies the
re-creation of the Perl interpreter for each data record, which can be
relatively time consuming, particularly for Perl scripts which
themselves perform a rapid operation per data record.
</p>
<p>
So, to avoid significant differences in performance timings between RTC
and non-RTC contexts, the Perl component's default behavior is to <i>keep</i>
the same instance of the Perl interpreter throughout its life cycle. This
is indicated by the False setting of the <em>Rebuild In RTC</em> parameter.
</p>
<p>
However, there are some circumstances where Perl script or 3rd party modules
are not written in a way that allows them to safely re-use the same interpreter in this way. So if you see unexpected behavior from a Perl
component in a Run To Completion subprotocol, switch the <em>Rebuild In RTC</em>
parameter to True to see if that resolves the issue.
</p>
<p>
If that fixes the problem, and the Perl cannot be re-cast to
handle the persistent interpreter,
you may need to trade performance for stability.
</p>
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Tue Feb 05 15:52:10 2013
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Get RID
390 -1355
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
<li> If you see unexpected or erroneous behavior when using this component
in a Run To Completion subprotocol, please look at the help text for the
<em>Rebuild In RTC</em> parameter, which is in the Implementation tab.</li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_comp_dev.pdf')">Perl Component Development Guide</a></li>
</ul>
Generic
Generic
Generic
The server version when this component was registered
9.0.1.15
Local Component Identifier
18
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
#timeOut:=False;
if StrLength(@'Time Out')>0 then
if (DateSecondsSince1970()-@StartTime)/60 > @'Time Out' then
#timeOut:=True;
end if;
end if;
if #timeOut then
False;
else
sleep(@'Polling Time' * 1000);
True;
end if;
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
@'Time Out':=Trim(@'Time Out');
@'Polling Time':=Trim(@'Polling Time');
if integer(@'Polling Time') <= 0 then
@'Polling Time' := 61; // default to poll no more than once a minute
end if;
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Timeout?
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
1290 -1355
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
14
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@BlastError:='No RID found. BLAST search failed.';
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
BLAST ERROR
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
690 -1535
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
20
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The names of properties to remove from each data record entering the component
responseStatus
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
the action to perform
Remove
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 07 13:23:27 2008
Remove Properties
Protocol AutoLayout
true
0
EXPRESSIONNODE
Component Revision
100
The component this component is based upon.
SubProtocol
Protocol Scale
0
1590 -1895
{C2ED35FF-5397-4877-AADB-10F1A15D60E2}
100
Removes the specified properties
For each data record that enters the component, the properties specified by <i>PropertyList</i> are removed from the record. To specify more than one property to be removed, use a comma separated list.
<p><b>See Also:</b></p>
<p>To keep only a few properties from a long list, it may be more efficient to instead use <a href="javascript:DoLink('{456195EF-9708-4367-B3DD-21EBF3796604}')">Keep Properties</a>.</p>
Generic
Generic
Data records with the specified properties removed
None
The server version when this component was registered
7.0.0.31
By Ref
0
Protocol Sticky Notes
Local Component Identifier
13
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Remove Properties
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@BlastError:= 'Online BLAST search timed out.';
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Time Out Error
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1590 -2255
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
15
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
StrLength(@EncodedQuery)>0;
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Any Query
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -815
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
24
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
When a General or DataType error occurs during processing:
LeaveData: The data is passed unalterd
ClearData: The data is cleared before being passed (an empty data item)
DeleteData: The data is deleted and not passed
LeaveData
ClearData
DeleteData
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
the action to perform
PassDataToPort
the port to pass data to
0
COMPONENTNODE
The user who registered this component.
scitegic/generic
The date this component was registered.
Wed Sep 20 08:16:06 2006
Don't Pass Data
The component this component is based upon.
Evaluate Expression
0
{F70D3185-8B78-4211-AF3E-D5427C306F87}
390 -995
100
Prevents records exiting an internal component from being passed out of a subprotocol
<p>This component is useful inside subprotocols to keep data from being passed out. It is an alternative to disabling components.</p>
<p><b>See Also:</b></p>
<p><a href="javascript:DoLink('scihelpdocs/user/comps/disable_comps.htm')">Disabling Components</a></p>
Generic
None
None
The server version when this component was registered
6.0.1.0
Local Component Identifier
25
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Dont Pass Data
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
@'XML Output File' ne ''
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentReturnsPass
ComponentReturnsFail
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Save XML Output File?
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
390 -1355
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
6
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
The source file path, relative to the server machine. This file is unaffected by the copy operation.
Internet files (http, https and ftp protocols) are supported as sources for copying. When copying from http and https URLs you may optionally specify Authentication Options for the connection in the appropriate group parameters.
Source Authentication Options
$(responseXML)
For http and https sources, you may optionally specify authentication information for the URL in the following parameters
Source Authentication Options Username
Source Authentication Options Password
Source Authentication Options Authentication Method
SetDisabledDetails('To enable, specify an HTTP URL for the Source');
Parameter('Source') rlike '^https?:';
Determines which method to use during authentication with an HTTP or HTTPS source.
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component Source parameter group's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. NOTE: if the <i>Username</i> parameter is provided, its value will override the user's logged in username.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used to connect to the supplied URL Source.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
must be filled in to log into the remote server.
</p>
<p>
If the Authentication Method is set to 'Use Pipeline Pilot Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username will be used along with the user's Pipeline Pilot password.
</p>
The password used to authenticate with the Source server. This parameter is disabled if Authentication Method is not set to 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set Authentication Method to 'Use Username and Password Parameters'");
Parameter('Source Authentication Options Authentication Method') eq 'Use Username and Password Parameters';
Al1HAAJHAlRoZSBkZXN0aW5hdGlvbiBmaWxlIHBhdGggcmVsYXRpdmUgdG8gdGhlIHNlcnZlciBtYWNoaW5lLiAgVGhpcyBmaWxlIG1heSBvciBtYXkgbm90IGV4aXN0IGJlZm9yZSB0aGUgY29weSBvcGVyYXRpb24uICBJZiBpdCBkb2VzIGV4aXN0IGFscmVhZHksIHRoZSBleGlzdGluZyBmaWxlIHdpbGwgYmUgb3ZlcndyaXR0ZW4gYnkgdGhlIGNvcGllZCBmaWxlLiBJZiBpdCBkb2VzIG5vdCBleGlzdCwgdGhlbiBhbnkgbmVjZXNzYXJ5IG5ldyBpbnRlcm1lZGlhdGUgZm9sZGVycyBhcmUgYWxzbyBjcmVhdGVkLgoKV2ViRGF2IHJlcG9zaXRvcnkgZmlsZXMgc3VjaCBhcyBmb3IgTWljcm9zb2Z0IFNoYXJlcG9pbnQgc2VydmVycyAodXNpbmcgaHR0cCBhbmQgaHR0cHMgcHJvdG9jb2xzKSBhcmUgc3VwcG9ydGVkIGFzIGRlc3RpbmF0aW9ucyBmb3IgY29weWluZy4gIFdoZW4gY29weWluZyB0byBhbiBodHRwIG9yIGh0dHBzIFVSTCwgeW91IG1heSBvcHRpb25hbGx5IHNwZWNpZnkgQXV0aGVudGljYXRpb24gT3B0aW9ucyBmb3IgdGhlIGNvbm5lY3Rpb24gaW4gdGhlIEF1dGhlbnRpY2F0aW9uIE9wdGlvbnMgcGFyYW1ldGVyIGdyb3VwLgETRGVzdGluYXRpb25VUkxUeXBlAAwAAAABAQAAARxUaGlzIHBhcmFtZXRlciBpcyByZXF1aXJlZC4EAfwAAAAAAAAAAAAwRGVzdGluYXRpb24DzFPPuwQA
DestinationURLType
LinkSingleParentParameterType
The destination file path relative to the server machine. This file may or may not exist before the copy operation. If it does exist already, the existing file will be overwritten by the copied file. If it does not exist, then any necessary new intermediate folders are also created.
WebDav repository files such as for Microsoft Sharepoint servers (using http and https protocols) are supported as destinations for copying. When copying to an http or https URL, you may optionally specify Authentication Options for the connection in the Authentication Options parameter group.
Destination Authentication Options
XML Output File
For http and https destinations, you may optionally specify authentication information for the URL in the following parameters
Destination Authentication Options Username
Destination Authentication Options Password
Destination Authentication Options Authentication Method
SetDisabledDetails('To enable, specify an HTTP URL for the Destination');
Parameter('Destination') rlike '^https?:';
Determines which method to use during authentication with an HTTP or HTTPS destination.
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component Source parameter group's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. NOTE: if the <i>Username</i> parameter is provided, its value will override the user's logged in username.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used to connect to the supplied URL Destination.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
must be filled in to log into the remote server.
</p>
<p>
If the Authentication Method is set to 'Use Pipeline Pilot Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username will be used along with the user's Pipeline Pilot password.
</p>
The password used to authenticate with the Destination server. This parameter is disabled if Authentication Method is not set to 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set the Destination Authentication Method to 'Use Username and Password Parameters'");
Parameter('Destination Authentication Options Authentication Method') eq 'Use Username and Password Parameters';
Componet Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
COMPONENTNODE
{38A20EDC-AAAA-42EF-97BA-D1B40B74FFE0}
The component this component is based upon.
Copy File
0
690 -1355
100
Copies a single file and saves it to the specified location. Can be used to retrieve files from HTTP or HTTPS servers and can also be used to upload files to HTTP WebDav repositories such as Sharepoint and other document repositories.
<p>A copy of <i>Source</i> is created and saved to <i>Destination</i>. The original file is left in place.</p>To copy multiple files, invoke the component once per file to be copied.</p>
<b>Notes:</b>
<ul>
<li>All file locations are relative to the server.</li>
<li>You can copy to FTP, HTTP and HTTPS servers using this component and to WebDav document repositories such as Sharepoint or Apache WebDav.</li>
<li>For HTTP and HTTPS servers, you can specify authentication options to control username and password information for the connection.</li>
</ul>
<b>See Also:</b>
<ul>
<li>To copy a file from the client to the server, use <a href="javascript:DoLink('{BBBC3189-A239-45BD-98A7-6FD3982D8282}')">Copy to Server</a>.</li>
<li>To copy a file from the server to the client, use <a href="javascript:DoLink('{DC38B2F0-B8B1-4114-92F0-5EE15A9E969C}')">Copy To Client</a>.</li>
<li>To move a file, use <a href="javascript:DoLink('{40A2ACDC-AABB-42EF-970A-D1340B748FE0}')">Move File</a>.</li>
</ul>
None
None
None
The server version when this component was registered
5.0.0.103
Local Component Identifier
5
The destination file, relative to the server
$(Destination)
The source file, relative to the server
$(Source)
Parameter Help
CopyFile
The user who registered this component.
scitegic/generic
The date this component was registered.
Tue Feb 23 10:56:01 2010
Copy File
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\File Utilities\Copy File
Al1HAAJIAVRoZSBuYW1lIG9mIHRoZSBjYWNoZSB0byByZWFkLCB3aGljaCBzaG91bGQgcmVmZXIgdG8gYSBjYWNoZSBhbHJlYWR5IHdyaXR0ZW4gaW4gdGhlIHByb3RvY29sIGNvbnRhaW5pbmcgdGhpcyBjb21wb25lbnQuICBVc2UgdGhlIDxhIGhyZWY9ImphdmFzY3JpcHQ6RG9MaW5rKCd7QjkxRUU1MjEtQzAyOC00NkJFLTg5QTQtNTFDREU5ODc3NTg5fScpIj5DcmVhdGUgVGVtcG9yYXJ5IENhY2hlSURzPC9hPiBjb21wb25lbnQgdG8gcHJvdmlkZSBhbiBJRCBmb3IgdGVtcG9yYXJ5IGNhY2hlcywgdGhlbiByZWZlciB0byB0aGUgdGVtcG9yYXJ5IElEIHdpdGggJChnbG9iYWxuYW1lKSxTdHJpbmdUeXBlAAwAAAABAQABAAQA/AAAAAAAAAAAACBDYWNoZUlEA8xTz7skc2VxQ2FjaGUA
StringType
StringType
The name of the cache to read, which should refer to a cache already written in the protocol containing this component. Use the <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a> component to provide an ID for temporary caches, then refer to the temporary ID with $(globalname)
RangeMinimum
RangeMaximum
Scope
JobId
SeqCache
The lower limit of data records to read from the cache. Numbering starts at 1. A blank value or a value less than 1 means start reading at the first cached record.
1
The upper limit of data records to read from the cache. Numbering starts at 1. A blank value or a value less than 1 means read to the end of the set of cached records.
The scope of the cache determines the length of time the cache exists and
the visibility of the cache to other jobs and users. Each scope has its own namespace, so you have to
specify both the <i>CacheID</i> and the <i>Scope</i> to the same values as were used in the </i>Cache Writer</i>.
Job Only
User Only
All Users
If True, then delete the cache when finished reading
True
False
Additional parameters to control the component behavior.
Read Globals
<p>Defines if the component should read globals from the cache.</p>
<p>Any globals read from the cache will overwrite the existing globals.</p>
True
False
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
If the scope is set to <I>Job Only</I>, then this is the job that owns the cache, if it is different from this job. The job must have been run by the same user as this job. If not specified, then the current job is assumed to contain the cache.
parameter('Scope') eq 'Job Only'
The cache type, that determines its folder location within the job.
Specifies the action of this component:
- Writer: The component writes data records to a new cache.
- Reader: The component reads data records from a pre-existing, completed cache.
- Writer/Reader: The component writes data out to a cache and then reads it in again to generate new records.
Writer
Reader
Writer/Reader
The user who registered this component.
scitegic/generic
The date this component was registered.
Fri Jun 25 19:09:54 2010
Cache Reader
990 -1535
DATABASENODE
{704E3781-33EA-42BB-9099-05B44C3D08E6}
The component this component is based upon.
Cache Reader
0
100
Reads data records from a named cache
<p>A data cache is a place where you can temporarily store data records without losing information. This component is used to read data from a named cache created by the <a href="javascript:DoLink('{674E3781-33EA-42BB-9099-49CB4C3D08E6}')">Cache Writer</a> component.</p>
<p>Use <i>CacheID</i> to specify the name of the cache to read.</p>
<b>Notes:</b>
<ul>
<li>This component is typically used as a mechanism for retrieving temporary data stored within a protocol.</li>
<li>To access a cache created in a different protocol, use <i>Scope</i>.</li>
</ul>
<b>See Also:</b>
<ul>
<li>To join data from a cache to streaming data records use <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a>.</li>
<li>To list available caches use <a href="javascript:DoLink('{704E3781-33EA-42BB-7077-05B44C3D08E6}')">List Caches</a>.</li>
<li>To delete a cache use <a href="javascript:DoLink('{704E3781-33EA-42BB-9088-05B44C3D08E6}')">Delete Cache</a>.</li>
<li>To create a temporary cacheID use <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a>.</li>
</ul>
None
Generic
None
The server version when this component was registered
7.5.1.5
Local Component Identifier
9
The original path within the database of this component.
Components\Data Access and Manipulation\File Readers\Cache Reader
Al1HAAEeRmlsZSBjb250YWluaW5nIEJMQVNUIHJlc3VsdHMgVVJMVHlwZQAMAAAAAQEAAQQEAfwAAAAAAAAAAAAcU291cmNlA8xTz7sBE3VzZXJkYXRhOi8zc2VxLnhtbAA=
URLType
URLType
File containing BLAST results
File Format
responseXML
The file format of the similarity search results file
blast
blastxml
fasta
gcg
hmmer
Al1HAAFCVHJ1bmNhdGUgdGhlIG51bWJlciBvZiBIU1BzIChoaWdoIHNjb3JpbmcgcGFpcnMpIHBlciBkYXRhYmFzZSBoaXQkTG9uZ1R5cGUACAAAAAEBAAEABAD8AAAAAAAAAAAAARVNYXhpbXVtIEhTUHMgUGVyIEhpdAPMU8+7EDEwMAA=
LongType
LongType
Truncate the number of HSPs (high scoring pairs) per database hit
Maximum HSPs Per Hit
Result Text Property
Use BioPerl BLAST Parser
BoolType
If True, BioPerl will be used to parse BLAST result file. If False and <i>File Format</i> is <b>blastxml</b>, a faster proprietary parser will be used.
Use BioPerl BLAST Parser
Name of property to hold raw result text. Property will be added to the SearchResult node.
SetDisabledDetails(
'To enable '.ParameterName().', set Use BioPerl BLAST Parser to "True"');
Parameter('Use BioPerl BLAST Parser');
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
0
DOCUMENTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{AD3A9F58-6E64-493A-ACE7-D1B759431A72}
1290 -1535
By Ref
1
100
Adds existing BLAST or HMMER results to a sequence or HMM
Useful for adding existing BLAST or HMMER results to a sequence or HMM in a protocol. There are parameters for the file location and file format.
<strong>See Also:</strong> <a href="javascript:DoLink('sciapps/scitegic/seqanal/docs/seqanal.pdf')">Sequence Analysis User Guide</a>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
17.2.0.1247
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/seqanal
The date this component was registered.
Tue Oct 4 08:28:54 2016
Join Data from Similarity Search Results File
Protocol AutoLayout
1
Local Component Identifier
31
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
The original path within the database of this component.
Components\Biology\Sequence Analysis\Search and Similarity\Manipulators\Join Data from Similarity Search Results File
An XPath query or queries to determine the nodes to apply the PilotScript to. For example, "//*[@ID]" would select all nodes with the property "ID" defined; if the Expression were "NewID := 'NewID_' . ID;", then a new property NewId would be created, and consist of the old ID prepended with the string NewId_.
/*/*/*
Expression (or list of expressions separated by semi-colons) to evaluate for each selected node
Initial Expression
Final Expression
remove('blastXMLOutputFile');
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
The name of the parser to use for the query. Depending on the choice of parser, different XPath features will be supported. Primarily, "Simple XPath" is fast but supports fewer features, and "Full XPath" is show but supports more XPath features.
Simple XPath
Full XPath
When a General or DataType error occurs during processing:
LeaveData: The data is passed unalterd
ClearData: The data is cleared before being passed (an empty data item)
DeleteData: The data is deleted and not passed
LeaveData
ClearData
DeleteData
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Apply PilotScript
<p>
Our view of the data record for use in the Select query.
</p>
Attributes
Elements
ComponentNode
The user who registered this component.
scitegic/generic
The date this component was registered.
Fri Aug 17 08:38:14 2012
Apply PilotScript to Selected Nodes
0
{AEE34E57-CBE3-4091-AE4A-2AF22B7CF406}
1590 -1535
100
Selects a set of nodes in the data record and applies the PilotScript expression to its properties.
Selects a set of nodes in the data record and applies a PilotScript expression to each.
For example, "//*[@ID]" would select all nodes with the property "ID" defined; if the Expression were "NewID := 'NewID_' . ID;", then a new property NewId would be created, and consist of the old ID prepended with the string NewId_.
Generic
Generic
None
Local Component Identifier
34
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Data Tree Manipulators\Apply PilotScript to Selected Nodes
9.0.0.8
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
if @SeqCount==1 and @'Result Text Property' ne '' then
Property(@'Result Text Property') := @resultText;
end if;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Add Result Text
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
1890 -1535
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
7
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
@resultFound == 1
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
Result?
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -1715
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
8
The original path within the database of this component.
Components\Data Access and Manipulation\Property Filters\Custom Filter (PilotScript)
7.0.0.35
The scope within which we wish to list all currently-existing caches.
<I>Job Only</I> lists caches scoped to the currently running protocol.
<I>User Only</I> lists all caches created so that all jobs by that user can see them.
<I>All Users</I> lists all caches created by any user in the global scope.
Job Only
User Only
All Users
Contains less commonly used parameters
Specific CacheID
Al1HAALgAElmIHNwZWNpZmllZCwgdGhlbiBvbmUgc2luZ2xlIGRhdGEgcmVjb3JkIGlzIG91dHB1dCBvbmx5IGlmIHRoYXQgZGF0YSBjYWNoZSBleGlzdHMgaW4gdGhlIHNjb3BlLCBvdGhlcndpc2Ugbm90aGluZyBpcyBvdXRwdXQuCgpJZiB0aGUgcGFyYW1ldGVyIGlzIG5vdCBzcGVjaWZpZWQsIHRoZW4gb25lIGRhdGEgcmVjb3JkIGZvciBlYWNoIGNhY2hlIGluIHRoZSBzY29wZSBpcyBvdXRwdXQuICAsU3RyaW5nVHlwZQAIAAAAAQEAAQAEAfwAAAAAAAAAAAABEVNwZWNpZmljIENhY2hlSUQDzFPPuyRzZXFDYWNoZQA=
StringType
StringType
If specified, then one single data record is output only if that data cache exists in the scope, otherwise nothing is output.
If the parameter is not specified, then one data record for each cache in the scope is output.
SeqCache
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
The cache type, that determines its folder location within the job.
Specifies the action of this component:
List
The user who registered this component.
scitegic/generic
The date this component was registered.
Wed Oct 07 10:46:20 2009
List Caches
390 -1895
DATABASENODE
{704E3781-33EA-42BB-7077-05B44C3D08E6}
The component this component is based upon.
List Caches
0
100
Lists all caches or determines if a specific cache ID exists in the given scope
This component lists all caches that exist in the given scope. A record is output for each cache found.
<b>Specific CacheID</b>: if filled in, then one single data record is output only if that data cache exists in the scope, otherwise nothing is output. Thus, this component can be used as a trigger for a downstream pipeline which is only executed if a particular cache exists.
The output data record contains these properties:
<dl>
<dt><i>CacheID</i>
<dd>The name of the cache
<dt><i>Scope</i>
<dd>The scope where the cache resides
<dt><i>NumRecords</i>
<dd>The number of records in the cache
<dt><i>NumBytes</i>
<dd>The number of bytes in all files used to store the cache data
<dt><i>Owner</i>
<dd>The user who created the cache
<dt><i>Date</i>
<dd>The date the cache was last modified
<dt><i>Expiration</i>
<dd>The date beyond which the administrator is free to remove this cache if he or she chooses.
</dl>
None
Generic
None
The server version when this component was registered
5.0.0.13
Local Component Identifier
27
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Cache Utilities\List Caches
Al1HAAJIAVRoZSBuYW1lIG9mIHRoZSBjYWNoZSB0byByZWFkLCB3aGljaCBzaG91bGQgcmVmZXIgdG8gYSBjYWNoZSBhbHJlYWR5IHdyaXR0ZW4gaW4gdGhlIHByb3RvY29sIGNvbnRhaW5pbmcgdGhpcyBjb21wb25lbnQuICBVc2UgdGhlIDxhIGhyZWY9ImphdmFzY3JpcHQ6RG9MaW5rKCd7QjkxRUU1MjEtQzAyOC00NkJFLTg5QTQtNTFDREU5ODc3NTg5fScpIj5DcmVhdGUgVGVtcG9yYXJ5IENhY2hlSURzPC9hPiBjb21wb25lbnQgdG8gcHJvdmlkZSBhbiBJRCBmb3IgdGVtcG9yYXJ5IGNhY2hlcywgdGhlbiByZWZlciB0byB0aGUgdGVtcG9yYXJ5IElEIHdpdGggJChnbG9iYWxuYW1lKSxTdHJpbmdUeXBlAAwAAAABAQABAAQA/AAAAAAAAAAAACBDYWNoZUlEA8xTz7skc2VxQ2FjaGUA
StringType
StringType
The name of the cache to read, which should refer to a cache already written in the protocol containing this component. Use the <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a> component to provide an ID for temporary caches, then refer to the temporary ID with $(globalname)
RangeMinimum
RangeMaximum
Scope
JobId
SeqCache
The lower limit of data records to read from the cache. Numbering starts at 1. A blank value or a value less than 1 means start reading at the first cached record.
1
The upper limit of data records to read from the cache. Numbering starts at 1. A blank value or a value less than 1 means read to the end of the set of cached records.
The scope of the cache determines the length of time the cache exists and
the visibility of the cache to other jobs and users. Each scope has its own namespace, so you have to
specify both the <i>CacheID</i> and the <i>Scope</i> to the same values as were used in the </i>Cache Writer</i>.
Job Only
User Only
All Users
If True, then delete the cache when finished reading
True
False
Additional parameters to control the component behavior.
Read Globals
<p>Defines if the component should read globals from the cache.</p>
<p>Any globals read from the cache will overwrite the existing globals.</p>
True
False
Component Attributes
ComponentReturnsPass
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
If the scope is set to <I>Job Only</I>, then this is the job that owns the cache, if it is different from this job. The job must have been run by the same user as this job. If not specified, then the current job is assumed to contain the cache.
parameter('Scope') eq 'Job Only'
The cache type, that determines its folder location within the job.
Specifies the action of this component:
- Writer: The component writes data records to a new cache.
- Reader: The component reads data records from a pre-existing, completed cache.
- Writer/Reader: The component writes data out to a cache and then reads it in again to generate new records.
Writer
Reader
Writer/Reader
The user who registered this component.
scitegic/generic
The date this component was registered.
Fri Jun 25 19:09:54 2010
Cache Reader
690 -1895
DATABASENODE
{704E3781-33EA-42BB-9099-05B44C3D08E6}
The component this component is based upon.
Cache Reader
0
100
Reads data records from a named cache
<p>A data cache is a place where you can temporarily store data records without losing information. This component is used to read data from a named cache created by the <a href="javascript:DoLink('{674E3781-33EA-42BB-9099-49CB4C3D08E6}')">Cache Writer</a> component.</p>
<p>Use <i>CacheID</i> to specify the name of the cache to read.</p>
<b>Notes:</b>
<ul>
<li>This component is typically used as a mechanism for retrieving temporary data stored within a protocol.</li>
<li>To access a cache created in a different protocol, use <i>Scope</i>.</li>
</ul>
<b>See Also:</b>
<ul>
<li>To join data from a cache to streaming data records use <a href="javascript:DoLink('{BDEEEC41-E62B-4FDE-B0D0-8B9C96CA2497}')">Join Data from Cache</a>.</li>
<li>To list available caches use <a href="javascript:DoLink('{704E3781-33EA-42BB-7077-05B44C3D08E6}')">List Caches</a>.</li>
<li>To delete a cache use <a href="javascript:DoLink('{704E3781-33EA-42BB-9088-05B44C3D08E6}')">Delete Cache</a>.</li>
<li>To create a temporary cacheID use <a href="javascript:DoLink('{B91EE521-C028-46BE-89A4-51CDE9877589}')">Create Temporary CacheIDs</a>.</li>
</ul>
None
Generic
None
The server version when this component was registered
7.5.1.5
Local Component Identifier
26
The original path within the database of this component.
Components\Data Access and Manipulation\File Readers\Cache Reader
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
blastError:= @BlastError;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
Custom Error Message
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Blast Error
Automatically update the component display name when the "Expression" property is changed. This feature is turned off after the user manually changes the display name of the component in Protocol workspace.
#displayName := trim(Parameter('Expression', 2));
if(strlength(#displayName) > 0) then
/* Find the first line break */
#nl := FindSubstring(#displayName, "\r");
if #nl <= 0 then
#nl := FindSubstring(#displayName, "\n");
end if;
/* Keep the first line only */
if (#nl > 0) then
StrErase(#displayName, #nl);
end if;
#displayName;
else
"";
end if;
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
990 -1895
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
Generic
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
29
The original path within the database of this component.
Components\Data Access and Manipulation\Property Manipulators\Custom Manipulator (PilotScript)
When a General or DataType error occurs during processing:
LeaveData: The data is passed unalterd
ClearData: The data is cleared before being passed (an empty data item)
DeleteData: The data is deleted and not passed
LeaveData
ClearData
DeleteData
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
When an expected data type is not found during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
When an recoverable error occurs during processing:
Halt: Terminate the execution of the network
Fail: Continue execution of the network, passing data out the Fail(red) port
Pass: Continue execution of the network, passing data out the Pass(green) port
Halt
Fail
Pass
CustomErrorText
the action to perform
PassDataToPort
the port to pass data to
2
FilterNode
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Mar 29 17:36:03 2004
Data to Fail Port
The component this component is based upon.
Evaluate Expression
0
{A718CDEE-EE7D-4B75-9346-C3780D347090}
1290 -1895
100
Directs records to the Fail port for use in subprotocol building.
Useful in a subprotocol for directing records that are passing through components in the subprotocol to the Fail port of the subprotocol.
Generic Data
None
Generic Data
All data records
The server version when this component was registered
4.0.4.0
Local Component Identifier
30
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Data to Fail Port
Al1HAAKEAFRoZSBuYW1lIG9mIHRoZSBjYWNoZSB0byBkZWxldGUsIHdoaWNoIHNob3VsZCByZWZlciB0byBhIGNhY2hlIGFscmVhZHkgd3JpdHRlbi4gSWYgdGhpcyBjYWNoZSBkb2VzIG5vdCBleGlzdCwgbm90aGluZyB3aWxsIGJlIGRvbmUuLFN0cmluZ1R5cGUADAAAAAEBAAEABAH8AAAAAAAAAAAAIENhY2hlSUQDzFPPuyRzZXFDYWNoZQA=
StringType
StringType
The name of the cache to delete, which should refer to a cache already written. If this cache does not exist, nothing will be done.
Scope
SeqCache
The scope of the cache determines the length of time the cache exists and
the visibility of the cache to other jobs and users. Each scope has its own namespace, so you have to
specify both the <i>CacheID</i> and the <i>Scope</i> to the same values as were used in
the </i>Cache Writer</i> to test for the existance of a particular cache.
Job Only
User Only
All Users
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
If the scope is set to <I>Job Only</I>, then this is the job that owns the cache, if it is different from this job. The job must have been run by the same user as this job. If not specified, then the current job is assumed to contain the cache.
The cache type, that determines its folder location within the job.
Specifies the action of this component:
Delete
The user who registered this component.
scitegic/generic
The date this component was registered.
Tue Apr 26 16:51:08 2005
Delete Cache
90 -2255
DATABASENODE
{704E3781-33EA-42BB-9088-05B44C3D08E6}
The component this component is based upon.
Delete Cache
0
100
Deletes a particular cache
This component deletes a cache. To fully specify a cache, you need the values of <I>CacheID</I> and <I>Scope</I> that were used when it was created using the <I>Cache Writer</I>.
<P>If the cache does not exist, this component does nothing. If it does exist, then the cache is deleted.</P>
None
None
None
The server version when this component was registered
5.0.0.13
Local Component Identifier
14
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Cache Utilities\Delete Cache
The path of the file or folder to be deleted. Can be a path on the server or a network resource.
<p>
<ul>
<li>For a file on or accessible to the server (e.g., via NFS), enter a path.</li>
<li>For a network resource, enter a network URL in the form <b>protocol://server:port/...</b>. (If port is omitted, a default is used.) Protocols supported are:
<ul>
<li>http</li>
<li>https</li>
<li>hdfs (Hadoop Distributed File System)</li>
</ul>
</li>
</ul>
Filename Authentication Options
Recursive
$(responseXML)
For http and https sources, you may optionally specify authentication information for the URL in the following parameters
Filename Authentication Options Username
Filename Authentication Options Password
Filename Authentication Options Authentication Method
Filename Authentication Options Additional Security Info
SetDisabledDetails('To enable, specify an HTTP URL for the Filename');
Parameter('Filename') rlike '^https?:';
Determines which method to use during authentication with an HTTP or HTTPS filename.
<ul>
<li><b>Use Username and Password Parameters</b>: Use the values provided by the component Filename parameter group's <i>Username</i> and <i>Password</i> parameters.
<li><b>Use Pipeline Pilot Credentials</b>: Use the username and password that were provided when the user logged into Pipeline Pilot. NOTE: if the <i>Username</i> parameter is provided, its value will override the user's logged in username.
</ul>
Use Username and Password Parameters
Use Pipeline Pilot Credentials
<p>
The username used to connect to the supplied URL Filename.
</p>
<p>
If the Authentication Method is set to 'Use Username and Password Parameters', then this parameter
must be filled in to log into the remote server.
</p>
<p>
If the Authentication Method is set to 'Use Pipeline Pilot Credentials', then this parameter can be left empty
and the logged in username is used. As an override option, if the username is filled in, this
username will be used along with the user's Pipeline Pilot password.
</p>
The password used to authenticate with the Filename server. This parameter is disabled if Authentication Method is not set to 'Use Username and Password Parameters'.
SetDisabledDetails("To enable, set Authentication Method to 'Use Username and Password Parameters'");
Parameter('Filename Authentication Options Authentication Method') eq 'Use Username and Password Parameters';
<p>This is an array of additional security information. Primarily used by HDFS. </p>
<p>When connecting to secured HDFS the Kerberos KDC Server and realm must be specified. </p>
<b>For example</b>
<p>KDC Server=<Server Name></p>
<p>KDC Realm=<Realm Name></p>
If the path provided for is a folder then if the value for recursive is true, then the sub-folders will also be deleted. If now value is not provided then it is assumed to be false. The value is ignored when the path provided is for a file.
True
False
Componet Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Parameter Help
DeleteFile
The user who registered this component.
scitegic/generic
The date this component was registered.
Thu Sep 29 11:14:09 2016
Delete File
ComponentNode
{38A20EDC-8D3C-42EF-97BA-D1B40B74FFE0}
The component this component is based upon.
Delete File
0
390 -2285
100
Deletes the specified file or folder from the Pipeline Pilot server, from a WebDav repository such as Sharepoint, for from Hadoop HDFS
<p>The file or folder specified by <i>Filename</i> is deleted.</p>
<b>Note:</b>
<ul>
<li>The file or folder location is relative to the server.</li>
<li>An HTTP or HTTPS filename can be used to delete files from a WebDav file repository such as Sharepoint</li>
<li>If an HTTP or HTTPS URL is used for the filename, authentication options can be filled in to provide username and password credentials to the remote server.
</ul>
None
None
None
Local Component Identifier
15
Parameter Help
$(Filename)
17.2.0.1279
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\File Utilities\Delete File
<p>
Path to a file on the server to open in Discovery Studio. You can use any file format supported by Discovery Studio.
</p>
$(RunDirectory)/$(SequenceFileName).xml
<p>Contains less commonly used parameters.</p>
Script Name
The name of a file which will be created in the Run Directory containing the script to open with Discovery Studio. If left as "ViewResults.ds_pl" the file will automatically be opened by Discovery Studio if the protocol is run as a Quick Launch protocol.
ViewResults.ds_pl
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
streamfile
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
MessageBox
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
VIEWERNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{FD223270-EA75-9A28-026C-3D48C2A5CF2D}
1290 -1715
By Ref
1
100
Displays a file in Discovery Studio
<p>
Path to a file on the server to open in Discovery Studio. You can use any file format supported by Discovery Studio.
</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
7.5.2.500
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Mon Sep 27 11:51:43 2010
DS File Viewer
Protocol AutoLayout
1
0
Local Component Identifier
39
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Chemistry\Data Access and Manipulation\Viewers\DS File Viewer
<p>Perl script to execute. Pipeline Pilot executes the following three subroutines:
<ul>
<li>onInitialize: the subroutine that executes before the first data record is processed.</li>
<li>onProcess: the subroutine that executes for each data record.</li>
<li>onFinalize: the subroutine that executes after the last data record has been processed.</li>
</ul>
<p>
The usual Perl scoping rules apply to variables. To pass a variable between the three required subroutines use package level variables, e.g., <code>$::variable</code> or <code>my $variable</code> declared outside the subroutines.
</p>
<p>
The <i>onInitialize</i> and <i>onFinalize</i> subroutines both receive the <code>pilot::Context Perl</code> object as the only argument. The <i>onProcess</i> subroutine receives the <code>pilot::Context</code> and <code>pilot::DataRecord</code> Perl objects as its two arguments.
</p>
<p>
pilot::Context provides access to globals and parameters. A common way to get these is via a Perl hash reference. For example, to get a <i>Source</i> parameter value use
</p>
<pre>
my $parametersHashRef = $context->getComponentParameters()->getHashRef();
my $source = $parametersHashRef->{"Source"};
</pre>
<p>
To obtain the value of a global property called <i>Old</i> and set a new global property called <i>New</i> use
</p>
<pre>
my $globalsHashRef = $context->getGlobalProperties()->getHashRef();
my $value = $globalsHashRef->{"Old"};
$globalsHashRef->{"New"} = "newValue";
</pre>
<p>
Top level properties in a data record are accessed similarly.
</p>
<pre>
my $node = $data->getRoot();
my $propertiesHashRef = $node->getProperties()->getHashRef();
my $value = $propertiesHashRef->{"Old"};
$propertiesHashRef->{"New"} = "newValue";
</pre>
<p>
<i>onInitialize</i> and <i>onProcess</i> must return one of the following component state values:
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
</p>
<p>
To explicitly route a data record to a port use
</p>
<pre>
$data->routeTo($pilot::PassPort);
</pre>
<p>
or
</p>
<pre>
$data->routeTo($pilot::FailPort);
</pre>
<p>
Data records can also be terminated internally by routing to <code>$pilot::NoPort</code>.
</p>
use strict;
use DS::Comm;
use Proteomics::BLASTCommon;
use DS::Report;
sub onInitialize {
my $context = shift;
return pilot::READYFORINPUTDATA;
}
sub onProcess {
my $context = shift;
my $data = shift;
return pilot::READYFORINPUTDATA;
}
sub onFinalize {
my $context = shift;
my $report = new DS::Report($context);
my $seqName = $context->getGlobalProperties->getHashRef->{SequenceName};
my $jobDir = $context->getGlobalProperties->getHashRef->{JobDir};
my $sf = $context->getGlobalProperties->getHashRef->{SequenceFileName};
my $xml = "$jobDir/$sf.xml";
my $numHits = BLASTCommon::numHit($xml);
if ($numHits) {
my $msg = "$seqName found $numHits NCBI BLAST hit".($numHits > 1 ? "s": "");
$report->addSummary($msg);
}
else {
my $msg = "No BLAST hits returned for $seqName";
$report->addSummary($msg);
}
my $msg = "$numHits hit".($numHits > 1 ? 's' : '').": $seqName";
DS::Comm::finalMessage($context, $msg);
}
The name of an application package that contains Perl modules referenced by this component. By defining this package name, a package-specific module may be referenced with the 'use modulename;' syntax.
scitegic/dscore
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
If True, the Perl interpreter instance for the component is terminated in the onFinalize event.
True
False
Halt
Fail
Pass
Halt
Fail
Pass
The user who registered this component.
scitegic/integration
The date this component was registered.
Mon Nov 06 19:45:37 2006
{6C9F19F9-6CF3-4BB9-8F61-89C89D9CDCBB}
Comment
Loads a class to provide component behavior.
Component Usage
Loads a class to provide component behavior.
Set the ports and icon as appropriate.
DS Report Summary and Final Message
90 -1895
COMPONENTNODE
The component this component is based upon.
Plug In
0
100
Executes Perl code on the server
<p>Allows a Perl script to manipulate a data record's property list, the global
property list, and component behavior. The <em>Script</em> parameter contains
the script that implements the component behavior, encoded in Perl. The following
subroutines must be defined in this text parameter (all three methods need to
be declared in the main scope): </p>
<ul>
<li><em>onInitialize: </em>Invoked once before the any data records
arrive. This method receives one argument, a pilot::Context object. Also,
it must return a component state, one of $pilot::ReadyForInputData, $pilot::DoneProcessingData, $pilot::ReadyForNewData or $pilot::ReadyForInputThenNewData. </li>
<li><em>onProcess</em>: Called once per data record. This method
receives two arguments, a pilot::Context and a pilot::DataRecord object and
return a component state. </li>
<li><em>onFinalize</em>: Called when no more data records are available.
This method receives one argument, a pilot::Context object. </li>
</ul>
<p><i>onInitialize</i> and <i>onProcess</i> must
return one of the following component state values:</p>
<ul>
<li>pilot::READYFORINPUTDATA
<li>pilot::DONEPROCESSINGDATA
<li>pilot::READYFORNEWDATA
<li>pilot::READYFORINPUTTHENNEWDATA
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li>Any script outside of the subroutines is executed just prior to calling
onInitialize.</li>
<li>The optional <em>Use Package</em> parameter is used to identify a package
that holds one or more Perl modules that are referenced by the component.
In this way, much or all of a component's functionality can be maintained
in a package module. This facilitates component upgrading, by centralizing
the code.</li>
<li>The Perl include path for the library must be declared in the package.conf
file for the named package.</li>
<li>Perl's <em>strict</em> and <em>warnings</em> directives can be very useful
when writing scripts. To detect warnings, you need to create a handler
to manage them, since by default, Perl sends them to STDERR, which is not
easily accessible from within Pipeline Pilot.
<p>For example, the following
script treats warnings as errors:</p>
<p><code>use warnings;<br />
$SIG{__WARN__} = sub {die @_;};</code></p></li>
</ul>
<p><strong>See Also:</strong></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_api.htm')">Perl Scripting Component API</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/integration/docs/server_side/perl_comp_dev.pdf')">Perl Component Development Guide</a></li>
</ul>
None
None
None
The server version when this component was registered
6.0.1.4
Local Component Identifier
33
The original path within the database of this component.
Components\Database and Application Integration\Application Integration\Programmatic Integration\Perl (on Server)
After displaying the molecules, pop up a message box. The message can explain the actions applied to the molecules.
$(ExperimentSummary)
<p>
Details to be shown if the user clicks the "MORE" button on the message box.
</p><p>
The details message can be plain text or html.
parameter('Message') ne ''
<p>
Text key used to disable showing messages user has already seen.
</p><p>
If blank, message is always shown.
</p>
parameter('Message') ne ''
<p>Contains less commonly used parameters.</p>
Script Name
See Also Report Link
In the "More" message, generate a "See Also" link to the report.
parameter('Message') ne ''
True
False
The name of a file which will be created in the Run Directory containing the script to open with Discovery Studio. If left as "ViewResults.ds_pl" the file will automatically be opened by Discovery Studio if the protocol is run as a Quick Launch protocol.
ViewResults.ds_pl
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
streamfile
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
MessageBox,ShowReport
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
VIEWERNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{4563A009-DAFE-4FBA-02E1-7E11F0BBA93F}
390 -1925
By Ref
1
100
Displays a message in Discovery Studio
<p>
A rich text message is displayed in Discovery Studio.
</p>
None
None
Records passing out of any unconnected pass ports of components inside the subprotocol
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
8.5.0.200
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Wed Nov 23 12:49:26 2011
DS Message Viewer
Protocol AutoLayout
1
0
Local Component Identifier
32
Subprotocol Made Link Into Copy
0
The original path within the database of this component.
Components\Chemistry\Data Access and Manipulation\Viewers\DS Message Viewer
<p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.</p>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.</p><p>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When <i>RunToCompletion</i> is set to "True - For Each Data Record", the subprotocol is re-initialized and executed in its entirety for each individual data record it processes.
<p>
When set to "True - For Each Category", the data records are divided into categories based on the value of the Categorize Using parameter and then the subprotocol is re-initialized and executed in its entirety for each category of data records. When used in conjunction with <i>Parallel Processing Options</i>, each category of data records will be sent to the remote server as part of one single batch.
RunToCompletion Categorize Using
False
True - For Each Data Record
True - For Each Category
Required when RunToCompletion is set to <i>True - For Each Category</i>, this parameter specifies the property or properties used to define the categories.
SetInvalidDetails('This parameter is required when RunToCompletion is set to True - For Each Category');
Parameter() ne '';
SetDisabledDetails('To Enable, Set RunToCompletion to True - For Each Category');
Parameter('RunToCompletion') eq 'True - For Each Category';
<p>
When set to True, this set of options allow the subprotocol to execute in parallel on one or more independent Pipeline Pilot servers, running local or remote. Additional parameters control the number of data records to send in each process batch, the list of Pipeline Pilot servers to use, the number of protocol processes that can be executed on each server, and whether to preserve the order of the processed data records.
</p><p>
Use this option judiciously, since it will not increase throughput in all circumstances. In general, it will yield improved performance when the parallelization benefits outweigh the data transfer overhead for the specified batch size. As a rough guide, performance can be improved when each individual batch requires at least 10 seconds to execute, although this can be affected by the size of each data record.
</p><p>
When running the protocol with debugging, the subprotocol will provide debugging messages to detail each of the various steps of execution. Also, components located within the parallel subprotocol will display their debugging messages as well.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>The number of records to send to the remote server(s) for each subprotocol execution. In general, performance is optimized when the amount of time required to process each batch is at least 10 seconds (note that this can vary depending on the characteristics of the remote server's hardware). Use a batch size of -1 to pass all data in a single batch.</p>
<p>When <i>RunToCompletion</i> is set to "True - for each category", the batch sizes will be determined by the number of records in each individual category. Therefore, the <i>Batch Size</i> parameter is not applicable.
#parallel := Parameter('Parallel Processing Options');
#rtcForEachCategory := (Parameter('RunToCompletion') eq 'True - For Each Category');
if (not #parallel) then
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
elsif (#rtcForEachCategory) then
SetDisabledDetails("This parameter is not applicable when 'RunToCompletion' is set to 'True - For Each Category'");
end if;
#parallel and not #rtcForEachCategory;
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
Parallel Processing Options Server Run On Grid
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Run subprotocol on grid engine if available.
Parallel Processing Options Server Run On Grid Queue Name
parameter('Parallel Processing Options');
True
False
Select the queue name on the grid to submit jobs to. Leave blank to use default queue name.
parameter('Parallel Processing Options Server Run On Grid');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Set Error Message
Protocol AutoLayout
1
0
SUBPROTOCOLNODE
Component Revision
790
{17001C74-63CE-4B4B-B37C-F0EAA19CE4AA}
1290 -2075
By Ref
0
100
Builds subprotocols in Pipeline Pilot.
<p>
Use a subprotocol to collapse one or more components into a single item. Open a subprotocol to view or edit its contents.
</p><p>
To define input and output ports, display icon, and parameter settings, right-click and select Edit. You can save subprotocols and reuse them. For more information, see the User Guide help for <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_overview.htm')">Subprotocols Overview</a>.
</p><p>
For more information on the parallel subprotocol options, see <a href="javascript:DoLink('scihelpdocs/user/subprots/subprot_parallel_processing.htm')">Parallel Processing Subprotocols</a>.
</p><p>
Tip: Any component can be made into a shortcut, so that the latest version is always used. For more information, see <a href="javascript:DoLink('scihelpdocs/user/subprots/shortcuts.htm')">Creating Shortcuts</a>.
</p>
Generic
Generic
Records passing out of any unconnected pass ports of components inside the subprotocol
Generic
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
7.9.0.30
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Owner Access Only
The user who registered this component.
scitegic/coreutils
The date this component was registered.
Tue Jan 26 10:02:05 2010
Local Component Identifier
40
The original path within the database of this component.
Components\Data Access and Manipulation\Utilities\Subprotocol Utilities\Subprotocol
The PilotScript expression used to filter the data.
Initial Expression
Final Expression
Keep Calculated Properties
exitStatus is defined and exitStatus > 0 and stderr is defined
<p>
An expression evaluated only when the component
is first initialized (when the first data record enters).
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
An expression evaluated only when the
component is finalized (once the last data
record has passed).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Allows any properties that were calculated in
"Expression" to be included with the exiting
record.
Otherwise, these properties are removed after
the evaluation is complete.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:57:16 2008
exitStatus is defined and exit
FILTERNODE
The component this component is based upon.
Custom Filter (PilotScript)
0
{DD58ECD1-516F-48FE-A5DB-8328AED680F2}
90 -95
100
Passes data records for which a PilotScript expression evaluates to True
<p>For each incoming data record, the PilotScript statements in <i>Expression</i> are evaluated. If the last statement in the expression evaluates to True or non-zero, the record is sent out the Pass port. Otherwise it is sent out the Fail port.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
</ul>
Generic
Generic
Data records for which the final expression in the PilotScript evaluates to True or non-zero
Generic
Data records for which the final expression in the PilotScript evaluates to False or zero
Local Component Identifier
0
Components\Data Access and Manipulation\Filters\Custom Filter (PilotScript)
7.0.0.35
Expression (or list of expressions separated by semi-colons) to evaluate.
Initial Expression
Final Expression
Keep Calculated Properties
@ExperimentError:= stderr;
<p>
This expression is executed only the first time data passes into the component.
<p/>
Since there is no data available at initialization time, only expressions referencing local or global properties are allowed.
</p>
<p>
This expression is executed only at finalization (after the last data passes out of the component).
<p/>
Since there is no data available at finalization time, only expressions referencing local or global properties are allowed.
</p>
Adds any properties calculated in this component to a data record's property list.
True
False
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
When TRUE, controls which exit port is used for
each data record by evaluating the final expression
(or only one) listed in the "Expression" parameter.
-Pass (green) port when expression true
-Fail (red) port when expression false
True
False
Version of expression parser to use.
2
The user who registered this component.
scitegic/generic
The date this component was registered.
Mon Jan 21 09:55:43 2008
Set Error Message
The component this component is based upon.
Custom Manipulator (PilotScript)
EXPRESSIONNODE
0
{0995D675-7EE2-432C-95DF-E91C14052B22}
378 -95
100
Applies a PilotScript expression to each data record
<p>PilotScript is the native scripting language in Pipeline Pilot. It is based on PL/SQL and allows you to query or alter the properties on data records. For example, the expression:
<blockquote>
<code>A := 10;</code>
</blockquote>
creates a property named "A" and sets the value to "10".</p>
<p>For each incoming data record the PilotScript statements in <i>Expression</i> are evaluated.</p>
<p><b>Notes:</b>
<ul>
<li>To initialize global variables such as counters, use <i>Initial Expression</i>.</li>
<li>To clean up global variables or calculate final results use <i>Final Expression</i>.</li>
<li>Parameters on Custom Manipulators do not become global variables that can be accessed via <i>@ParameterName</i> syntax. They must be accessed by using the <i>Parameter('ParameterName')</i> function or by using token substitution via <i>$(ParameterName)</i> syntax.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li>For more information on PilotScript, see <a href="javascript:DoLink('scihelpdocs/user/pscript/pscript_overview.htm')">PilotScript Overview</a>.</li>
<li>To use PilotScript to filter data records use <a href="javascript:DoLink('{DD58ECD1-516F-48FE-A5DB-8328AED680F2}')">Custom Filter (PilotScript)</a>.</li>
</ul>
Generic
None
None
The server version when this component was registered
7.0.0.35
Local Component Identifier
1
Components\Data Access and Manipulation\Manipulators\Custom Manipulator (PilotScript)
Comma separated list of filenames in the Output directory.
File Name Ignore
$(SequenceFileName).xml,$(SequenceFileName)_profile.pir,blastseqs.fasta, HspLigands.txt, HspSCOPs.txt, PDBResolution.txt
Comma separated list of file names which won't show up in the DS Report.
A comma separated list of help text that must match the above file sequnce. The specified text will be used in the Report.
File Name Help Sort
BLAST Hits, Multiple Sequence Alignment of Query to Hits, Full Hit Sequences, Ligand Contacts, SCOP Codes,X-ray Resolutions
List the files sorted by the names specified in <em>File Name Help</em>. If disabled, the files are listed in the order specified in <em>File Name</em>.
parameter('File Name Help') ne ''
True
False
Advanced settings.
Help Text Results
View Results File Name
Post Process HTML Files
This text will be displayed in the Results section of the report. The text can contain html tags.
Name of the file that will be linked to the "View Results Script" button in the report.
ViewResults.ds_pl
<p>A comma separated list of names that must match the file sequence in <i>File Name</i>. The specified names will be used in the report. If an empty field is specified, the base-filename will be used.</p>
#A1 := parameter();
Expand(',',#A1);
#A2 := parameter('File Name');
Expand(',',#A2);
#N1 := NumValues(#A1);
#N2 := NumValues(#A2);
SetInvalidDetails(
ParameterName() . ' must have same number of entries as: File Name'.
" ". #N1. " vs. ". #N2 );
not ( #N1 > 0 and #N1 != #N2)
</p>A comma delimited list of global properties, each of which will automatically hold the name of a file in the job's temporary folder, whose name is guaranteed to be unique.<p/>For each global name, an additional global is created (with the suffix <i>_Filename</i>) that hold just the leaf file name of the temporary file.<p/>
<p>Lists a set of properties (without the @) to create on the global property list for this subprotocol. Such properties can only be accessed within the scope of this subprotocol and its descendent subprotocols.<p/>Any global property not declared in this way is created on the top level global property list and is accessible from all protocols, but may collide with another property of the same name.<p/>Scoping global properties to the subprotocol where they are used is highly recommended.</p>
When RunToCompletion is set to True, the subprotocol is re-initialized and executed in its entirety for each record it processes.
True
False
<p>Execute this protocol in parallel on Pipeline Pilot servers. You can set the number of data records sent in each process batch, the name of the servers, the number of protocol processes executed on each server, and whether to preserve the order of the processed data records.</p>
<p><strong>Notes</strong>:</p>
<ul>
<li>Not all parts of the protocol are run in parallel, which may affect scaling. In general, performance improves when the parallelization benefits outweigh the data transfer overhead for the specified batch size.</li>
<li>By default, Pipeline Pilot servers are configured to allow a maximum of four processes per protocol. You can adjust this on the Pipeline Pilot Server Administration Portal.</li>
</ul>
Parallel Processing Options Batch Size
Parallel Processing Options Server
Parallel Processing Options Preserve Order
True
False
<p>Execute this protocol in parallel on Pipeline Pilot servers. You can set the number of data records sent in each process batch, the name of the servers, the number of protocol processes executed on each server, and whether to preserve the order of the processed data records.</p>
<p><strong>Notes</strong>:</p>
<ul>
<li>Not all parts of the protocol are run in parallel, which may affect scaling. In general, performance improves when the parallelization benefits outweigh the data transfer overhead for the specified batch size.</li>
<li>By default, Pipeline Pilot servers are configured to allow a maximum of four processes per protocol. You can adjust this on the Pipeline Pilot Server Administration Portal.</li>
</ul>
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
25
<p>
Comma separated list of Pipeline Pilot servers on which to execute the parallel subprotocol. The server names should be in the same format as the Pipeline Pilot Client's change server dialog (server-name:port-number).
</p><p>
Specifying <b>localhost</b> as a server will cause the subprotocol to use its own Pipeline Pilot server. This can be helpful on multi-CPU/multi-core systems (where the subprotocol jobs are spawned on the same machine) and Pipeline Pilot Linux clusters (where 'localhost' requests spawning the subprotocol jobs on the same cluster).
</p>
Parallel Processing Options Server Processes
SetInvalidDetails('This parameter is required.');
Parameter() ne '';
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Comma separated list of maximum number of batches that can be simultaneously executed on each remote server. The number of entries in this parameter must equal the number of servers. If '*' is specified as the number of processes, then that server will only launch as many batches as the administrator of the remote server has permitted.
/* Server Processes must have the same number of comma separated entries as the Server parameter and must have a value*/
#processes := parameter();
expand(',', #processes);
#servers := parameter('Parallel Processing Options Server');
expand(',', #servers);
if (numvalues(#processes) == 0) then
SetInvalidDetails('This parameter is required.');
else
SetInvalidDetails('The number of server processes (' . numvalues(#processes) . ') must equal the number of servers (' . numvalues(#servers) . ').');
end if;
numvalues(#processes) > 0 and numvalues(#processes) == numvalues(#servers);
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
Whether to preserve the order of the data as it enters and leaves the remote subprotocol.
SetDisabledDetails("This parameter is only applicable if 'Parallel Processing Options' is 'True'");
Parameter('Parallel Processing Options');
True
False
This group contains parameters specific to protocols containing Reporting components.
Reporting Stylesheet
Specifies the name of a Stylesheet to be used by this protocol if any of the Reporting components have a Style selected in the <i>Format</i> and <i>Position</i> parameters.
{42691EC8-0CE0-4DB9-8B3C-5379CBB967A7}
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
<p>Post process HTML files by removing box-sizing script.</p>
True
False
Component Comment
Defines a subprotocol in a pipeline. When you are creating a pipeline that requires a subprotocol as a component, drag this component into the pipeline, open it, and add components to it.
Component Usage
When you are creating a pipeline which requires a subprotocol as
a component, you can drag this SubProtocol component into the
pipeline, open it, and add components into it.
Generate DS Report
Protocol AutoLayout
1
Protocol ShowComment
true
Add Protocol Comment Here
Add Protocol Comment Here
0
REPORTNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{E9901D25-DA09-42BA-9035-60CC818CA09D}
90 -2285
By Ref
1
100
Generates a Discovery Studio report
<p>Generates a report of a Discovery Studio protocol run.</p>
<p>
The report may contain the following sections:
</p>
<ul>
<li><strong>Description</strong>: Protocol help text.</li>
<li><strong>Information</strong>: Run-time information and status.</li>
<li><strong>Errors</strong>: Error messages created.</li>
<li><strong>Warnings</strong>: Warning messages created.</li>
<li><strong>Summary</strong>: Summary of the protocol run.</li>
<li><strong>Details</strong>: Additional information of the run.</li>
<li><strong>Results</strong>: Hyperlinks to result files created.
</li>
<li><strong>Parameters</strong>: Parameter settings used and hyperlinks to input files.
</li>
</ul>
<p><strong>Notes:</strong></p>
<ul>
<li> This component should be placed at the end of the protocol to ensure the output files are picked up correctly and the timing information is accurate.
</li>
<li>Help text can be added to the output files in the Results section.
</li>
</li>
<li>Any property values or reporting objects are added to the <strong>Summary</strong> section.
</li>
<li>The <a href="javascript:DoLink('{8F288684-93BC-4712-BC9A-73930339BDEB}')">DS Messaging</a> component can be used to add reporting text.</li>
</ul>
</ul>
None
Nore or more data records with properties or reporting objects which will be added to the Summary
Generic
A reporting object
None
Records passing out of any unconnected fail ports of components inside the subprotocol
The server version when this component was registered
9.0.0.100
Protocol Sticky Notes
Protocol Scale
100
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/dscore
The date this component was registered.
Wed Nov 21 08:37:38 2012
Local Component Identifier
26
Subprotocol Made Link Into Copy
0
Subprotocol Modified
0
The original path within the database of this component.
Components\Chemistry\Utilities\Discovery Studio\Generate DS Report
Title
UsingParent
<p>The optional title of the HTML document. The title is also used as part of the report's file name.</p>
Report
Include Format and Position CSS
Use Default Browser
Set IE Compatibility
Include Format and Position CSS
UsingParent
If set to <strong>False</strong>, all format and position settings will be omitted from the output document, allowing them to be set using an external stylesheet specified with an <a href="javascript:DoLink('{6DF7D4FA-28BC-453b-8882-DE55C7CFFBF9}')">Include</a>.
True
False
Set IE Compatibility
UsingParent
<p>When this flag is set to True (default), IE compatibility in the report is set to IE 10 (the minimum support level for Pipeline Pilot). For most users of IE, this should provide a stable environment to view Pipeline Pilot reports.
</p>
</p>Switch off this setting, if necessary, to allow Internet Explorer to display content in the highest mode available. This is only required for users who view the report in Internet Explorer 11 or above and who need to enable facilities not available in IE 10 mode, such as WebGL and some HTML5/CSS features.
</p>
<p>Note that even if you are viewing the report in another browser, other users may subsequently run the protocol or view the report in Internet Explorer, and so you may still need to set this IE Compatibility setting - but only in the rare case that your report requires something beyond the default IE 10 compatibility.
</p>
True
False
UsingParent
Browser
<p>
This parameter is only relevant when running the protocol in the Pipeline Pilot Professional Client.
</p>
<p>
The value specifies whether to use your default desktop web browser to view the report. When set to False, Internet Explorer is used as a standard fallback option. If IE is your default browser, it is used in either case.
</p>
<p>
If you use Internet Explorer, note that its behavior can be modified by switching off the <i>Set IE Compatibility</i> parameter. This is only relevant if you are working with more recently-supported web technologies, such as WebGL, that require at least IE 11 compatibility (and also require that IE 11 is installed, of course).
</p>
True
False
<p>Allows some alternative formatting and position settings to be used that may cause the HTML report output to behave and look more like the HTML generated with previous versions of Pipeline Pilot. This parameter is intended to be set on writers in protocols developed with previous versions of Pipeline Pilot where the output looks or behaves differently when run with the current version.</p>
<p><strong>Always use the value of "Default" for new protocols or protocols under active development. The latest Reporting components may not behave correctly in HTML reports written with a non-default mode.</strong></p>
<p>When set to "Pipeline Pilot 7.5", this parameter rolls back some of the improvements made in the name of standards compliance and formatting consistency, including:</p>
<ul>
<li>The report will not include the HTML 4.01 Doctype declaration, which will cause Internet Explorer to render the report using quirks mode.</li>
<li>Position rules will not be strictly enforced, so Internet Explorer 6 will render borders and padding outside the specified width.</li>
<li>Horizontal alignment of block objects will not be supported, so you will not be able to, for example, center-align a table.</li>
<li>The page margins will not be set.</li>
</ul>
Default
Pipeline Pilot 7.5
Lists the set of temporary files which will be converted by the network to true temporary file names
Lists a set of properties (without the @) to be created on the property list for this protocol.
Other properties are created on the global property list
and so are accessible from all protocols.
ServerFilePath
When the network is run, whether we process once or until done for each process call
True
False
The location for Internet Explorer
internetexplorer.application
The location for Chrome
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"
The location for Firefox
"C:\Program Files (x86)\Mozilla Firefox\firefox.exe"
Component Attributes
ComponentTakesInput
ComponentReturnsPass
ComponentReturnsFail
ComponentRunsLocal
Halt
Fail
Pass
Halt
Fail
Pass
DrainTopDown
Protocol Options
Make Shortcut by Default
Owner Access Only
The user who registered this component.
scitegic/report
The date this component was registered.
Thu Oct 22 23:43:53 2015
HTML Report Viewer
Protocol AutoLayout
1
0
REPORTVIEWERNODE
Component Revision
100
The component this component is based upon.
SubProtocol
{0952D9DF-F95C-4D43-826F-C70450811DD8}
390 -2285
By Ref
1
100
Displays the results of a reporting protocol in HTML format
<p>This component accepts reporting objects as input (e.g., tables, text and images), and displays the output as an HTML file in Internet Explorer.</p>
<p><b>Notes:</b></p>
<ul>
<li>Use the <a href="javascript:DoLink('{A28703A7-E701-4D34-9A93-1D24AC2039CC}')">Page</a> component to set the report's margins, borders, padding, and background formatting, or to specify HTML attributes on the report's <code><body></code> tag.</li>
</ul>
<p><b>See Also:</b></p>
<ul>
<li><a href="javascript:DoLink('sciapps/scitegic/report/docs/reporting_basic.pdf')">Report Design Essentials</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/report/docs/reporting_charts.pdf')">Chart Design</a></li>
<li><a href="javascript:DoLink('sciapps/scitegic/reportadv/docs/reportadv_interactive.pdf')">Interactive Reporting and Form Design</a></li>
</ul>
Generic
This component accepts reporting objects as input.
None
None
Protocol Sticky Notes
The server version when this component was registered
16.1.0.1089
Protocol Scale
100
Protocol Scale
0
Local Component Identifier
27
Subprotocol Made Link Into Copy
0
Output contains default meta header <meta http-equiv="X-UA-Compatible" content="IE=8">. This parameter can set different IE version. Valid value could be major version number of IE like "7", "9" or "EmulateIE7", "EmulateIE8", etc.
Subprotocol Modified
0
The original path within the database of this component.
Components\Reporting and Visualization\Reporting\Viewers\HTML Report Viewer
Component Icon
PROTOCOLNODE
Protocol Sticky Notes
%VERSION 3%
400 -219 1222 221
250 120 20
WARNING! This is a Pipeline Pilot protocol written in version 8.0 or later. It contains features that are not supported in earlier versions of Pipeline Pilot.