lua-users home
lua-l archive


Wim Couwenberg wrote:
Anyway, here's a simplistic
script to test binary-ness.  Adjust the pattern in "find" to something
more sensible, if you like.  Usage:

    lua isbin.lua <file-name>

file isbin.lua:

local now = os.clock()

local input, err =[1], "rb")
assert(input, err)

local isbin = false
local chunk_size = 2^12
local find = string.find
local read =

        local chunk = read(input, chunk_size)
        if not chunk then break end

        if find(chunk, "[^\f\n\r\t\032-\128]") then
                isbin = true
until false


now = os.clock() - now

if isbin then
        print "this file is binary..."
        print "this is a text file..."

print(string.format("this took %.3f seconds", now))

end of file

Woah, so non-English text files are binary? ;-)
(Perhaps by old FTP and Mail standards...)

Also, I believe \127 is seen as binary (DEL code), and \128 is already in high-Ascii area...
So perhaps I would rewrite your pattern as: [^\f\n\r\t\032-\126\192-\256]

Note I excluded the \128-\191 area, seen in ISO (8859-1 for example) as control characters, but if you consider the quite common Windows Ansi encoding (CP1252), it contains many valid characters, including the euro symbol, (c), (R), etc.

And, of course, your test doesn't work for UTF-8 and most other Unicode encodings. But that's another can of worms...

Additional note: many implementations of this kind of test agree that testing the first bytes (256, 512...) of a file is enough to see if it is binary or not. Perhaps it is too simplistic for some file formats, but it can work most of the time.

And I doubt there are so many text files of over 1GB... Except perhaps some exceptional log files or XML data files.

Philippe Lhoste
--  (near) Paris -- France