Difference between revisions of "Text Filters"

From TED Notepad
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<noinclude>{{manversion|5.0.0.13}}__NOTOC__</noinclude>
+
<noinclude>{{manversion|6.0.0.17|feature}}{{manversion|6.0.0.17|menu}}__NOTOC__</noinclude>
  
====Inspiration====
+
<q>How about a plugin architecture? Something simple - just a list of menu names and programs to run and maybe an indication of whether it should prompt for additional parameters. TED will run the program with those parameters and with current selection and then accept the output back into the TED workspace.</q> &nbsp; &nbsp; &nbsp; &nbsp; <i>(--- anonymous user ---)</i>
  
<q>How about a plugin architecture? Something simple - just a list of menu names and programs to run and maybe an indication of whether it should prompt for additional parameters. TED will run the program with those parameters and with the actual selection and then accept the output back into the TED workspace.</q> &nbsp; &nbsp; &nbsp; &nbsp; <i>(--- anonymous user ---)</i>
+
====Managing filters====
  
====Architecture====
+
Filters can be added thru the {{dialog|Settings}} dialog. See chapter [[Filters page]] of the {{dialog|Settings}} dialog for details on adding, modifying and managing filters.
  
Each {{feature|filter}} must be defined as a runnable command, consisting of an application file to execute and its parameters. System commands may be executed only through a {{system|command interpreter}} (i.e. {{system|cmd.exe}}, {{system|command.com}}, {{system|launch.exe}}) as described below.
+
====Technical details====
  
Each {{feature|filter}} can define up to 9 {{field|variables}} in the command. Use batch variable syntax (i.e. {{string|%1}}, {{string|%2}}, ..., {{string|%9}}) to define those {{field|variables}} in the command. When the {{feature|filter}} is to be used, a dialog will appear to ask for values of the {{field|variables}}. Furthermore, special {{string|%F}} variable can be used to refer to an actual path and file name of the current document. This can be used to run an external viewer (e.g. Internet Explorer or Firefox).
+
Upon {{feature|Text Filters|filter}} being executed, a new external process or command is started. Current selection is written into the {{system|stdin}}<sup>1</sup> of that process, and its {{system|stdout}}<sup>2</sup> and {{system|stderr}}<sup>3</sup> are read. Depending on what return code the process returns, the selection is either replaced with the output of the process, or an error message is displayed, containing returned error code and contents of the {{system|stderr}}.
 +
 
 +
: <small><sup>1</sup> {{system|standard console input}}</small>
 +
: <small><sup>2</sup> {{system|standard console output}}</small>
 +
: <small><sup>3</sup> {{system|standard console error output}}</small>
 +
 
 +
It is also possible to launch the filter with current selection as an argument, instead of pushing text into {{system|stdin}}. One can easily launch a browser and google the selected text for example.
 +
 
 +
Alternativelly, file name of the current document can be provided to the external filter. An external file viewer can be set this way and launched whenever necessary.
 +
 
 +
Finally, launch-time variables are available, thus the user can specify additional arguments upon each invocation of a filter. This can be useful for external greps, directory listings, etc.
 +
 
 +
====Filter commands====
 +
 
 +
Each {{feature|Text Filters|filter}} must be defined as a runnable command, consisting of an application file to execute and its parameters. System commands can be executed as well, but only through a {{system|command interpreter}}, i.e. {{system|cmd.exe}}, {{system|command.com}} or {{system|launch.exe}}, as described below.
 +
 
 +
Each {{feature|Text Filters|filter}} can use the following user and internal variables:
 +
 
 +
* {{syntax|Text Filters|%1}}, {{syntax|Text Filters|%2}}, ..., {{syntax|Text Filters|%9}} variables allow the user to specify launch-time command arguments upon each individual invocation, using the {{dialog|Filter}} dialog.
 +
* {{syntax|Text Filters|%F}} variable can be used to insert path and file name of the current document into the command. The document is saved automatically before the execution of such filter.
 +
* {{syntax|Text Filters|%S}} variable can be used to insert current selection into the command.
 +
* {{syntax|Text Filters|%%}} can be used to safely insert common {{string|%}} character into the command.
 +
 
 +
All command variables can be auto-enclosed with &quot;double-quotes&quot; by adding an ''escaping flag'' to them. Escaping flag is one of following:
 +
* Tilde (~) &mdash; Encloses the variable with double-quotes, and doubles all double-quotes in the variable. This is more common for native Windows commandline applications.
 +
** E.g. Variable {{string|%~1}} expands phrase {{string|Use "double-quotes" in parameters}} into {{string|"Use ""double-quotes"" in parameters"}}.
 +
* Back-slash (\) &mdash; Encloses the variable with double-quotes, and escapes all double-quotes in the variable with back-slashes. This is more common for unix and linux commandline applications.
 +
** E.g. Variable {{string|%\1}} expands phrase {{string|Use "double-quotes" in parameters}} into {{string|"Use \"double-quotes\" in parameters"}}.
 +
 
 +
Note: Unescaped variable, e.g. {{syntax|Text Filters|%S}}, does not add any extra characters to the command. This can be useful and sometimes even necessary, but it can also become unreliable, if spaces and other special characters are expected in such variable. Consider escaping variables, if spaces are expected to be given upon using a filter.
  
 
{{example start}}Example:
 
{{example start}}Example:
{{example body}}<nowiki>
+
{{example body}}<nowiki>myapp.exe %1
myapp.exe %1
 
  
myapp.exe /a %1 /b /c&quot;%2&quot;
+
myapp.exe /a %1 /b /c %2
  
myapp.exe /a %1 /b /c&quot;%1&quot;
+
myviewer.exe %~F
  
myviewer.exe /a %1 /b &quot;%F&quot;</nowiki>
+
grep.exe -e %\S</nowiki>
 
{{example end}}
 
{{example end}}
  
====Technology====
+
====Filter options====
 +
 
 +
For full list of all options a filter can employ, please see chapter [[Filters page]] of the {{dialog|Settings}} dialog. Following tips describe some of the options a filter can employ.
 +
 
 +
{{tip}} If the filter is expected to work with UTF-8 input and output, use the {{field|Settings:Filters:Filter|UTF-8}} option to get the selection automatically converted upon input and output.
 +
 
 +
{{tip}} It is possible to skip the launch dialog altogether upon invoking a filter by using {{field|Settings:Filters:Filter|Skip launch dialog}} option. This can be useful, if no variables and arguments are expected from the user anyway and the dialog just annoys the user.
 +
 
 +
{{tip}} Variable {{syntax|Text Filters|%F}}, along with {{field|Settings:Filters:Filter|Push input}} and {{field|Settings:Filters:Filter|Capture output}} options turned off, can be useful to define an external viewer which does not read the {{system|stdin}} at all, but requires a file name to start on. Also, {{field|Settings:Filters:Filter|Ignore return code}} can be useful if you don't want the external viewer to block the document while viewing. Please note that using the {{syntax|Text Filters|%F}} automatically saves the document before launching the filter as described below.
  
When a {{feature|filter}} is to be executed, a new process is started using the command of that {{feature|filter}}. All defined {{field|variables}} are replaced with values. The selection is then given to the {{system|stdin}}<sup>1</sup> of the started process. After the process returns, its {{system|stdout}}<sup>2</sup> or {{system|stderr}}<sup>3</sup> is read, depending on what value the process returned. Finally, the selection is replaced with the output of the process or an error is displayed, containing the returned error code and the content of the {{system|stderr}}<sup>3</sup>. All inputs/outputs are Non-UNICODE.
+
{{tip}} Certain filters might need to use the {{field|Settings:Filters:Filter|Mix stdout/stderr}} option, especially if the filter's {{system|stderr}} is a native part of the overall results.
  
If the {{string|%F}} variable is used, the document is automatically saved before the execution of the filter and the {{string|%F}} is replaced by the actual path and file name of the current document. The selection is ignored (neglected), which means, that nothing is written to filter's {{system|stdin}} and no output is awaited from filter's {{system|stdout}}. {{string|%F}} variable can be useful to define an external viewer which does not read the {{system|stdin}} at all but requires a file name to start on.
+
{{tip}} It is possible to display the filters' results in a message box instead of replacing the selection, using the {{field|Settings:Filters:Filter|Display as message}} option.
  
: <small><sup>1</sup> {{system|standard console input}}</small>
+
{{tip}} Some filters might make good use of the {{field|Settings:Filters:Filter|Exit when done}} option. The document is checked for saving before such exit.
: <small><sup>2</sup> {{system|standard console output}}</small>
 
: <small><sup>3</sup> {{system|standard console error output}}</small>
 
  
====Requirements====
+
====Filter requirements====
  
An application defined in {{feature|filter}}'s command should read the {{system|stdin}} for input text and should output any results to the {{system|stdout}}. If the operation is successful, the application should return zero. Otherwise it should return non-zero value and optionally describe the error to the {{system|stderr}}.
+
An application defined in {{feature|Text Filters|filter}}'s command should read the {{system|stdin}} for input text and should output any results to the {{system|stdout}}. If the operation is successful, the application should return zero. Otherwise it should return non-zero value and optionally describe the error thru the {{system|stderr}}.
  
Note: Some languages (like Pascal) does not support {{system|stderr}} at all. Therefore, if a non-zero return code is received and {{system|stderr}} is empty, {{system|stdout}} is expected to contain the error description.
+
Note: Some languages (like Pascal) do not support {{system|stderr}} at all. Therefore, if a non-zero return code is received and {{system|stderr}} is not used, {{system|stdout}} is expected to contain the error description message.
  
If an application file name ends with an {{string|.exe}} extension, it is not necessary to use it in the command, but it may be necessary otherwise. The command can specify the full path and file name of the application to execute as well as only the file name. In the case of a missing path, system uses the current working directory and system directories when searching for the application.
+
If an application file name ends with an {{string|.exe}} extension, it is not necessary to use it in the command, but it may be necessary otherwise. The command can specify the full path and file name of the application to execute as well as only the file name. In the case of a missing path, system uses the current working directory and system directories upon searching for the application.
  
 
System commands need to be executed through a {{system|command interpreter}}, as they do not have any file names to be executed. Use standard {{system|command interpreters}} (i.e. {{system|cmd.exe}} on Win NT/2000/XP, {{system|command.com}} on Win 9x/ME) or the {{system|launch.exe}} interpreter, which is included in the installation package and installed with TED Notepad.
 
System commands need to be executed through a {{system|command interpreter}}, as they do not have any file names to be executed. Use standard {{system|command interpreters}} (i.e. {{system|cmd.exe}} on Win NT/2000/XP, {{system|command.com}} on Win 9x/ME) or the {{system|launch.exe}} interpreter, which is included in the installation package and installed with TED Notepad.
  
 
{{example start}}Examples:
 
{{example start}}Examples:
{{example body}}
+
{{example body}}<nowiki>cmd.exe /c command [parameters]
cmd.exe /c command [parameters]
 
 
cmd /c dir /b %1
 
cmd /c dir /b %1
  
Line 51: Line 84:
  
 
launch.exe command [parameters]
 
launch.exe command [parameters]
launch dir /b %1
+
launch dir /b %1</nowiki>
 
{{example end}}
 
{{example end}}
  
 
====How to create a filter's application?====
 
====How to create a filter's application?====
  
Very easy! Any programming language that can be compiled to 16-bit/32-bit console application (i.e. PE files with an .exe extension) may be used (like c/c++, pascal, qbasic, ...). Such application should use its {{system|stdin}} for input, its {{system|stdout}} for output and its {{system|stderr}} in the case of failure/error (which are used by default in these languages). It should return zero upon success or non-zero value upon failure. No other requirements/restrictions are specified.
+
Very easy! Any programming language that can be compiled to a console application (i.e. PE files with an .exe extension) may be used, e.g. C/C++, Pascal, Visual Basic, etc. Even scripting languages are supported by using their associated interpreter for execution, e.g. PHP, Python, Perl, etc.
  
Note: Some languages (like Pascal) do not define any {{system|stderr}}. Therefore, all error messages could also be written to the {{system|stdout}}. When a non-zero error code is returned and no {{system|stderr}} is started, the {{system|stdout}} is supposed to be used instead of the {{system|stderr}} for any error messages to be displayed to the user.
+
Such application should use its {{system|stdin}} for input, its {{system|stdout}} for output and its {{system|stderr}} in the case of failure/error. It should return zero upon success and non-zero value upon failure. No other requirements are necessary. Applications can execute their own sub-processes, and even show user dialogs if necessary.
  
Note: Actually, both {{system|stdout}} and {{system|stderr}} are displayed to the user upon a non-zero error code. This behaviour may change in the future, according to the suggestions of the end users. The {{system|stderr}} is and will be preffered.
+
Note: Some languages do not define {{system|stderr}}. Therefore, all error messages can also be written to {{system|stdout}}. If a non-zero error code is returned and no {{system|stderr}} is started, {{system|stdout}} is read instead of {{system|stderr}} for any error messages to be displayed to the user.
  
 
{{example start}}Example:
 
{{example start}}Example:
Line 92: Line 125:
  
 
   return 0;                                  // ...success
 
   return 0;                                  // ...success
}</nowiki>
+
}
 +
</nowiki>
 
{{example end}}
 
{{example end}}
  
{{example start}}Example: (There is no stderr in Pascal, therefore writting errors to stdout too...)
+
{{example start}}Example: There is no {{system|stderr}} in Pascal, therefore writing errors to {{system|stdout}}...
 
{{example body}}<nowiki>
 
{{example body}}<nowiki>
 
var lines, code: integer;
 
var lines, code: integer;
Line 118: Line 152:
 
     lines:= lines - 1;
 
     lines:= lines - 1;
 
   end;
 
   end;
end. { ...autosuccess }</nowiki>
+
end. { ...autosuccess }
 +
</nowiki>
 
{{example end}}
 
{{example end}}
 +
 +
====Further details====
 +
 +
Note: Since version 6.3, any binary characters (i.e. the null character) are replaced with spaces before being sent to a filter.
 +
 +
Note: Since version 6.3, all newlines are converted to Windows type (i.e. {{string|CR/NL}}, i.e. {{string|13 10}} in ASCII, i.e. {{string|0D 0A}} in hex) before being sent to a filter.

Latest revision as of 23:07, 5 October 2021

This section is up to date for TED Notepad version 6.3.1.0.
Control page Control:feature:Text Filters
This section is up to date for TED Notepad version 6.3.1.0.
Control page Control:menu:Text Filters

How about a plugin architecture? Something simple - just a list of menu names and programs to run and maybe an indication of whether it should prompt for additional parameters. TED will run the program with those parameters and with current selection and then accept the output back into the TED workspace.         (--- anonymous user ---)

Managing filters

Filters can be added thru the Settings dialog. See chapter Filters page of the Settings dialog for details on adding, modifying and managing filters.

Technical details

Upon filter being executed, a new external process or command is started. Current selection is written into the stdin1 of that process, and its stdout2 and stderr3 are read. Depending on what return code the process returns, the selection is either replaced with the output of the process, or an error message is displayed, containing returned error code and contents of the stderr.

1 standard console input
2 standard console output
3 standard console error output

It is also possible to launch the filter with current selection as an argument, instead of pushing text into stdin. One can easily launch a browser and google the selected text for example.

Alternativelly, file name of the current document can be provided to the external filter. An external file viewer can be set this way and launched whenever necessary.

Finally, launch-time variables are available, thus the user can specify additional arguments upon each invocation of a filter. This can be useful for external greps, directory listings, etc.

Filter commands

Each filter must be defined as a runnable command, consisting of an application file to execute and its parameters. System commands can be executed as well, but only through a command interpreter, i.e. cmd.exe, command.com or launch.exe, as described below.

Each filter can use the following user and internal variables:

  • %1, %2, ..., %9 variables allow the user to specify launch-time command arguments upon each individual invocation, using the Filter dialog.
  • %F variable can be used to insert path and file name of the current document into the command. The document is saved automatically before the execution of such filter.
  • %S variable can be used to insert current selection into the command.
  • %% can be used to safely insert common % character into the command.

All command variables can be auto-enclosed with "double-quotes" by adding an escaping flag to them. Escaping flag is one of following:

  • Tilde (~) — Encloses the variable with double-quotes, and doubles all double-quotes in the variable. This is more common for native Windows commandline applications.
    • E.g. Variable %~1 expands phrase Use "double-quotes" in parameters into "Use ""double-quotes"" in parameters".
  • Back-slash (\) — Encloses the variable with double-quotes, and escapes all double-quotes in the variable with back-slashes. This is more common for unix and linux commandline applications.
    • E.g. Variable %\1 expands phrase Use "double-quotes" in parameters into "Use \"double-quotes\" in parameters".

Note: Unescaped variable, e.g. %S, does not add any extra characters to the command. This can be useful and sometimes even necessary, but it can also become unreliable, if spaces and other special characters are expected in such variable. Consider escaping variables, if spaces are expected to be given upon using a filter.

Example:
myapp.exe %1 myapp.exe /a %1 /b /c %2 myviewer.exe %~F grep.exe -e %\S

Filter options

For full list of all options a filter can employ, please see chapter Filters page of the Settings dialog. Following tips describe some of the options a filter can employ.

Tip: If the filter is expected to work with UTF-8 input and output, use the UTF-8 option to get the selection automatically converted upon input and output.

Tip: It is possible to skip the launch dialog altogether upon invoking a filter by using Skip launch dialog option. This can be useful, if no variables and arguments are expected from the user anyway and the dialog just annoys the user.

Tip: Variable %F, along with Push input and Capture output options turned off, can be useful to define an external viewer which does not read the stdin at all, but requires a file name to start on. Also, Ignore return code can be useful if you don't want the external viewer to block the document while viewing. Please note that using the %F automatically saves the document before launching the filter as described below.

Tip: Certain filters might need to use the Mix stdout/stderr option, especially if the filter's stderr is a native part of the overall results.

Tip: It is possible to display the filters' results in a message box instead of replacing the selection, using the Display as message option.

Tip: Some filters might make good use of the Exit when done option. The document is checked for saving before such exit.

Filter requirements

An application defined in filter's command should read the stdin for input text and should output any results to the stdout. If the operation is successful, the application should return zero. Otherwise it should return non-zero value and optionally describe the error thru the stderr.

Note: Some languages (like Pascal) do not support stderr at all. Therefore, if a non-zero return code is received and stderr is not used, stdout is expected to contain the error description message.

If an application file name ends with an .exe extension, it is not necessary to use it in the command, but it may be necessary otherwise. The command can specify the full path and file name of the application to execute as well as only the file name. In the case of a missing path, system uses the current working directory and system directories upon searching for the application.

System commands need to be executed through a command interpreter, as they do not have any file names to be executed. Use standard command interpreters (i.e. cmd.exe on Win NT/2000/XP, command.com on Win 9x/ME) or the launch.exe interpreter, which is included in the installation package and installed with TED Notepad.

Examples:
cmd.exe /c command [parameters] cmd /c dir /b %1 command.com /c command [parameters] command.com /c dir /b %1 launch.exe command [parameters] launch dir /b %1

How to create a filter's application?

Very easy! Any programming language that can be compiled to a console application (i.e. PE files with an .exe extension) may be used, e.g. C/C++, Pascal, Visual Basic, etc. Even scripting languages are supported by using their associated interpreter for execution, e.g. PHP, Python, Perl, etc.

Such application should use its stdin for input, its stdout for output and its stderr in the case of failure/error. It should return zero upon success and non-zero value upon failure. No other requirements are necessary. Applications can execute their own sub-processes, and even show user dialogs if necessary.

Note: Some languages do not define stderr. Therefore, all error messages can also be written to stdout. If a non-zero error code is returned and no stderr is started, stdout is read instead of stderr for any error messages to be displayed to the user.

Example:
#include <stdio.h> #include <stdlib.h> int main (int argc, char* argv[]) { int lines; char buff[1024]; if (argc < 2) { fprintf (stderr, "Enter the number!"); // outputs to stderr return 1; // ...failure! } lines = atoi (argv[1]); // gets 1st parameter and // converts it to integer if (lines < 1) { fprintf (stderr, "Wrong number!"); // outputs to stderr return 2; // ...failure! } while (lines != 0) { if (EOF == scanf ("%s", buff)) // reads from stdin break; // or breaks on end-of-input printf ("%s\n", buff); // writes to stdout lines = lines - 1; } return 0; // ...success }
Example: There is no stderr in Pascal, therefore writing errors to stdout...
var lines, code: integer; buff: string; begin if ParamCount < 1 then begin writeln ('Enter the number!'); { outputs } halt (1); { ...failure! } end; Val(ParamStr(1), lines, code); { gets 1st param and } { converts it to integer } if lines < 1 then begin writeln ('Wrong number!'); { outputs } halt (2); { ...failure! } end; while lines <> 0 do begin readln (buff); { reads a line from stdin } if eof then break; writeln (buff); { writes to stdout } lines:= lines - 1; end; end. { ...autosuccess }

Further details

Note: Since version 6.3, any binary characters (i.e. the null character) are replaced with spaces before being sent to a filter.

Note: Since version 6.3, all newlines are converted to Windows type (i.e. CR/NL, i.e. 13 10 in ASCII, i.e. 0D 0A in hex) before being sent to a filter.