| Following is the example of a program that accepts an arbitrarily long text as input and produces a list of words and their frequency of occurrence as output.
(The program is in ICON** V. 9.0 programming language)
** = Icon is a very high level general-purpose programming language with extensive features for processing strings (text) and data structures. Icon is an imperative, procedural language with a syntax that is reminiscent of C and Pascal, but with semantics at a much higher level.
global uses, lineno, width
procedure main(args)
width := 15 # width of word field
uses := table()
lineno := 0
every tabulate(words()) # tabulate all citations
output() # print the citations
end
# Add line number to citations for word
#
procedure tabulate(word)
/uses[word] := set()
insert(uses[word], lineno)
return
end
# Generate words
#
procedure words()
while line := read() do {
lineno +:= 1
write(right(lineno, 6), " ", line)
map(line) ? while tab(upto(&letters)) do {
s := tab(many(&letters))
if *s >= 3 then suspend s# skip short words
}
}
end
TOP
# Print the results
#
procedure output()
write() # blank line
uses := sort(uses, 3) # sort citations
while word := get(uses) do {
line :=""
numbers := sort(get(uses))
while line ||:= get(numbers) || ","
write(left(word, width), line[1:-2])
}
end
The program reads a line, writes it out with an identifying line number, and processes every word in the line. Words less than three characters long are considered to be "noise" and are discarded. A table, uses, is keyed by the words. Every key has a corresponding set of line numbers. The first time a word is encountered, a new set is created for it. The line number is inserted in any event. Since a value can be in a set only once, duplicate line numbers are suppressed automatically.
After all the input has been read, the table of words is sorted by key. Each corresponding set of line numbers is sorted and the line numbers are appended to the line to be written.
For example, if the input file is
On the Future!-how it tells
Of the rapture that impells
To the swinging and the ringing
Of the bells, bells, bells-
Of the bells, bells, bells, bells,
Bells, bells, bells-
To the rhyming and the chiming of the bells!
the output is
1 On the Future!-how it tells
2 Of the rapture that impells
3 To the swinging and the ringing
4 Of the bells, bells, bells-
5 Of the bells, bells, bells, bells,
6 Bells, bells, bells-
7 To the rhyming and the chiming of the bells!
and 3, 7
bells 4, 5, 6, 7
chiming 7
future 1
how 1
impells 2
rapture 2
rhyming 7
ringing 3
swinging 3
tells 1
that 2
the 1, 2, 3, 4, 5, 7 |
