Following is the example of
a program that accepts an arbitrarily
long text as input and produces
a list of words and their frequency
of occurrence as output.
(The program is
in ICON** V. 9.0 programming
language)
** = Icon is a
very high level general-purpose
programming language with extensive
features for processing strings
(text) and data structures.
Icon is an imperative, procedural
language with a syntax that
is reminiscent of C and Pascal,
but with semantics at a much
higher level.
global uses, lineno,
width
procedure main(args)
width := 15 #
width of word field
uses := table()
lineno := 0
every tabulate(words())
# tabulate all citations
output() # print
the citations
end
# Add line number
to citations for word
#
procedure tabulate(word)
/uses[word] :=
set()
insert(uses[word],
lineno)
return
end
# Generate words
#
procedure words()
while line :=
read() do {
lineno +:= 1
write(right(lineno,
6), " ", line)
map(line) ? while
tab(upto(&letters)) do {
s := tab(many(&letters))
if *s >= 3
then suspend s# skip short words
}
}
end
# Print the results
#
procedure output()
write() # blank
line
uses := sort(uses,
3) # sort citations
while word :=
get(uses) do {
line := ""
numbers := sort(get(uses))
while line ||:=
get(numbers) || ", "
write(left(word,
width), line[1:-2])
}
end
The program reads
a line, writes it out with an
identifying line number, and
processes every word in the
line. Words less than three
characters long are considered
to be "noise" and
are discarded. A table, uses,
is keyed by the words. Every
key has a corresponding set
of line numbers. The first time
a word is encountered, a new
set is created for it. The line
number is inserted in any event.
Since a value can be in a set
only once, duplicate line numbers
are suppressed automatically.
After all the
input has been read, the table
of words is sorted by key. Each
corresponding set of line numbers
is sorted and the line numbers
are appended to the line to
be written.
For example, if
the input file is
On the Future!-how
it tells
Of the rapture
that impells
To the swinging
and the ringing
Of the bells,
bells, bells-
Of the bells,
bells, bells, bells,
Bells, bells,
bells-
To the rhyming
and the chiming of the bells!
the output is
1 On the Future!-how
it tells
2 Of the rapture
that impells
3 To the swinging
and the ringing
4 Of the bells,
bells, bells-
5 Of the bells,
bells, bells, bells,
6 Bells, bells,
bells-
7 To the rhyming
and the chiming of the bells!
and 3, 7
bells 4, 5, 6,
7
chiming 7
future 1
how 1
impells 2
rapture 2
rhyming 7
ringing 3
swinging 3
tells 1
that 2
the 1, 2, 3, 4,
5, 7
