AWK: Get/Extract String Matching Regexp

Example of how to use AWK in order to get/extract/return parts of lines/strings, matching a "dynamic" regular expression (specified through a shell environment variable).

Source file used in this example

$ cat example.txt

Attr[27716, Field[4937190,usage=51]]

Attr[27716, Field[4937191,usage=10]]

Attr[27716, Field[4937192,usage=321]]

Two usages

# Defining the regular expression pattern as environment variable

$ MY_PATTERN="([0-9]+),used"

# Using the predefined awk variables RSTART and RLENGTH

# Returns the whole string matching the regexp

$ awk -v PATTERN="${MY_PATTERN}" 'match($0, PATTERN) { print substr( $0, RSTART, RLENGTH) }' example.txt

4937190,usage

4937191,usage

4937192,usage

# Using the capture group & array functionality in order to get only the regexp part in parenthesis

$ awk -v PATTERN="${MY_PATTERN}" 'match($0, PATTERN, array) { print array[1] }' example.txt

4937190

4937191

4937192

Using sed to replace parts of specific lines in a file

In this example we are using the following test file test.tmp (here listed, part of it) where we will replace the Oracle DB Version (the 11.2.0.4.0 below) between the path (/u00/app/oracle/product/) and . P0_EE in the line for DB  DB00001 by a new version string "--.-.-.-.-"

#SID;ORACLE_HOME;Position;Type;Standby;Cluster;DB-instances

. . .

DB00001;/u00/app/oracle/product/11.2.0.4.0.P0_EE;60;I;N;S;

DB00002;/u00/app/oracle/product/11.2.0.4.0.P0_EE;70;I;N;S;

This tests are performed under Linux in the bash shell.

In order to be flexible, we are using the following variables:

OS> DB=DB00001

OS> V=--.-.-.-.-

Here the sed command with the regular expression:

OS> sed 's|\('${DB}';/u00/app/oracle/product/\).*\(.P0_EE\)|\1'${V}'\2|g' test.tmp

The output:

#SID;ORACLE_HOME;Position;Type;Standby;Cluster;DB-instances

. . .

DB00001;/u00/app/oracle/product/--.-.-.-.-.P0_EE;60;I;N;S;

DB00002;/u00/app/oracle/product/11.2.0.4.0.P0_EE;70;I;N;S;

Explanation of the regular expression:

The first character after "s" (here the "|") defines the parts separator.

(Slashes (/) are commonly used, but since a big part of our expression is a UNIX path containing slashes, I decided to use another character in order to simplify the interpretation.)

The first part of the expression between the first two "|" (i.e. \('${DB}';/u00/app/oracle/product/\)\(.*\)\(. P0_EE\) ) defines what must match.

The escaped brackets "\(" and "\)" define back references that are referenced by \1 and \3 in the second part of the e.

The value of the variable ${DB} is concatenated with ";/u00/app/oracle/product/ ", forming the first "back reference block" that is referenced by the \1 after the "|".

The expression ".* " matches anything in between blocks one and two. It is not referenced, since if will be replaced by "${V}".

The second "back reference block" (.i.e. "\(.P0_EE\)") matches only ".P0_EE". It is referenced by the \2.

So, in our example, only lines matching the regular expression "DB00001;/u00/app/oracle/product/.*P0_EE" are changed and replaced by the string specified through "\1'${V}'\2" (between the second and third "|").

Another variant - more general and simpler - of the command (ignoring the ".P0_EE" part):

OS> sed 's|\('${DB}';/u00/app/oracle/product/\)\([^;]*\)|\1'${V}'|g' test.tmp

Output:

#SID;ORACLE_HOME;Position;Type;Standby;Cluster;DB-instances

. . .

DB00001;/u00/app/oracle/product/--.-.-.-.-;60;I;N;S;

DB00002;/u00/app/oracle/product/11.2.0.4.0.P0_EE;70;I;N;S;

January 20, 2017 at 10:47AM