Take the 2-minute tour ×

I'm trying to find a way to replace spaces and double quotes with pipes (||) while leaving the spaces within the double quotes untouched.

For example, it would make something like 'word "word word" word' into 'word||word word||word' and another like 'word word word' into 'word||word||word'.

Right now I have this to work off of:

[%- MACRO typestrip(value) PERL -%]
my $htmlVal = $stash->get('value');
$htmlVal =~ s/"/||/g;
print $htmlVal
[%- END -%]

Which handles replacing double quotes with pipes just fine.

I don't know how simple or complex this should be or if it can even be done, since I have no actual background in programming and, while I have worked with some Perl, it's never been this kind before, so I apologize if I'm not doing a good job of explaining this.

share|improve this question

4 Answers 4

up vote 0 down vote accepted

Seems possible and might be useful if only a regex is applicable:

 $htmlVal =~ s/(?:"([^"]+)"(\s*))|(?:(\S+)(\s*))/($1||$3).($2||$4?'||':'')/eg;

(Might be beautified a bit after closer introspection.)


 my $htmlVal ='word "word word" word';


 word||word word||word

Original code has been modified after failing this case:

 my $htmlVal ='word "word word" "word word"';

will now work too:

 word||word word||word word


 $htmlVal =~ s/
               (?: " ([^"]+) " (\s*)) # search "abc abc" ($1), End ($2)
               |                      # OR
               (?: (\S+) (\s*))       # abcd ($3), End ($4)
               ($1||$3) . ($2||$4 ? '||' : '') # decide on $1/$2 or $3/$4 



share|improve this answer
if it had this problem, there will be others. woudln't you rather use a parser? –  Joel Berger Jun 27 '12 at 20:42
@JoelBerger Yes, of course, sth. like this is not advisable for the average coder because he usually has no time to work this out. This is where it would be a responsible decision to use proven tech like a good parser. On the other side - he asked specifically for a regex - and he got one. I read the Friedl book (I bought all three as dead-tree versions) and just wanted to use the knowledge (Hey Jeffrey, are you here?) If you look at the /x-version of the regex, it seems easy enough to be probably of some general value (imho). –  rubber boots Jun 27 '12 at 20:51
Yeah ... nooooo ... use Text::ParseWords::shellwords which is tailor made for this situation. –  Sinan Ünür Jun 28 '12 at 13:07

I think it might be easier to use the core module Text::ParseWords to split on non-quoted whitespace, then rejoin the "words" with pipes.

#!/usr/bin/env perl

use warnings;
use strict;

use Text::ParseWords;

while (my $line = <DATA>) {
  print space2pipes($line); 
  print "\n";

sub space2pipes {
  my $line = shift;
  chomp $line;
  my @words = parse_line( qr/\s+/, 0, $line );
  return join '||', @words;

word "word word" word
word word word

Putting this into your templating engine is left as an exercise for the reader :-)

share|improve this answer

This is related to a frequently-asked question, answered in section 4 of the Perl FAQ.


Several modules can handle this sort of parsing—Text::Balanced, Text::CSV, Text::CSV_XS, and Text::ParseWords, among others.

Take the example case of trying to split a string that is comma-separated into its different fields. You can’t use split(/,/) because you shouldn’t split if the comma is inside quotes. For example, take a data line like this:



my @new = ();
           # groups the phrase inside the quotes
           | ([^,]+),?
           | ,

If you want to represent quotation marks inside a quotation-mark-delimited field, escape them with backslashes (e.g., "like \"this\"").


use Text::ParseWords;

For parsing or generating CSV, though, using Text::CSV rather than implementing it yourself is highly recommended; you’ll save yourself odd bugs popping up later by just using code which has already been tried and tested in production for years.

Adapting the technique to your situation gives


my @chunks;
push @chunks, $+ while $htmlVal =~ m{
  | (\S+)

$htmlVal = join "||", @chunks;
print $htmlVal, "\n";


word||word word||word

Looking back, it turns out that this is an application of Randal’s Rule, as dubbed in Regular Expression Mastery by Mark Dominus:

Randal's Rule

Use capturing or m//g when you know what you want to keep.

Use split when you know what you want to throw away.

Randal Schwartz

In your situation, you know what you want to keep, so use m//g to hang on to the text within quotes or otherwise separated by whitespace.

share|improve this answer
an interesting read, but doesn't it just say, "do what Joel said"? :-) –  Joel Berger Jun 27 '12 at 20:40
Agreed. "Do what Joel said!" ;-) but I do appreciate perlfaq links because it reminds people what a treasure trove of information can be found there. –  Sinan Ünür Jun 28 '12 at 13:05
Hear Hear! Remember, when in doubt: perldoc -q <searchterm>. (In this case perldoc -q split worked nicely) –  Joel Berger Jun 28 '12 at 14:20

While Joel's answer is fine, things can be simplified a bit by specifically using shellwords to tokenize lines:

#!/usr/bin/env perl

use strict; use warnings;
use Text::ParseWords qw( shellwords );

my @strings = (
    'word "word word" word',
    'word "word word" "word word"',

@strings = map join('||', shellwords($_)), @strings;

use YAML;
print Dump \@strings;

Isn't that more readable than a bunch of regex-gobbledygook?

share|improve this answer
Isn't that more readable than a bunch of regex-gobbledygook? - No, it is comparable if you are familiar w/regular expressions ;-) –  rubber boots Jun 28 '12 at 13:18
Ok fine, do what Sinan said. :-P –  Joel Berger Jun 28 '12 at 14:17
@JoelBerger This order is for the O.P., for Sinan, or for me? –  rubber boots Jun 28 '12 at 19:08
@rubberboots, it was more of a joke, we do basically the same thing. His function (shellwords) is a special version of parsewords for using space as the separator. –  Joel Berger Jun 28 '12 at 22:20

Your Answer


