org.licas_text.filter

Class LinearCounts

java.lang.Object

org.licas_text.filter.LinearCounts

public class LinearCounts
extends java.lang.Object

The default implementation of a linear count analysis model.

Author:

Kieran Greer

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.ArrayList	analysisOptions List of optional analysis operations
protected org.ai_heuristic.model.BagOfWords	bagOfWords Bag of words list.
protected boolean	compareWithFirst If true, only compare other analyses with the first document's analysis
protected int	maxNestingSequenceNumber The maximum number of words in the sequences to output to
protected int	minNestingSequenceNumber The minimum number of words in the sequences to output to
protected java.util.ArrayList<java.util.ArrayList<java.lang.String[]>>	popularSequences Popular sequences list.
protected int	popularSequencesNumber The number of sequences to output for each sequence number
protected java.util.ArrayList<java.lang.String[]>	popularWords List of most popular words only.

Constructor Summary

Constructors

Constructor and Description
LinearCounts() Create a new instance of LinearCounts.

Method Summary

Modifier and Type	Method and Description
protected void	createBoW(java.lang.String theText, boolean wordStem) Sort all words in the text to determine the most popular ones.
java.util.ArrayList<java.lang.String[]>	getPopularWords() Get the popular words list.
java.util.ArrayList<java.util.ArrayList<java.lang.String[]>>	getPopularWordSequences() Get the popular word sequences list.
protected java.lang.String	sequenceKey(java.lang.String sequence, boolean wordStem) Create a word sequence key, that might be different if word stemming is used.
void	sortMostPopularSequences(java.util.ArrayList<java.lang.String> singleWordsList, java.lang.String analysisTerm, int minNestingSequenceNumber, int maxNestingSequenceNumber, int popularSequencesNumber, boolean wordStem) Sort all words in the text to determine the most popular sequences.
void	sortMostPopularSequences(java.lang.String theText, java.lang.String analysisTerm, int minNestingSequenceNumber, int maxNestingSequenceNumber, int popularSequencesNumber, boolean wordStem) Sort all words in the text to determine the most popular sequences.
void	sortWordOrder(java.lang.String theText, int popularWordsNumber, int minWordLength, boolean wordStem) Sort all words in the text to determine the most popular ones.
void	sortWordOrder(java.lang.String theText, java.lang.String analysisTerm, int minWordLength, boolean wordStem) Sort all words in the text to determine the most popular ones.
protected void	sortWordOrder(java.lang.String theText, java.lang.String analysisTerm, int popularWordsNumber, int minWordLength, boolean wordStem) Sort all words in the text to determine the most popular ones.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

minNestingSequenceNumber

protected int minNestingSequenceNumber

The minimum number of words in the sequences to output to

maxNestingSequenceNumber

protected int maxNestingSequenceNumber

The maximum number of words in the sequences to output to

popularSequencesNumber

protected int popularSequencesNumber

The number of sequences to output for each sequence number

compareWithFirst

protected boolean compareWithFirst

If true, only compare other analyses with the first document's analysis

bagOfWords

protected org.ai_heuristic.model.BagOfWords bagOfWords

Bag of words list. List of arrays storing word name and count

popularWords

protected java.util.ArrayList<java.lang.String[]> popularWords

List of most popular words only. List of arrays storing word name and count

popularSequences

protected java.util.ArrayList<java.util.ArrayList<java.lang.String[]>> popularSequences

Popular sequences list. List of list of arrays storing word sequence and count

analysisOptions

protected java.util.ArrayList analysisOptions

List of optional analysis operations

Constructor Detail

LinearCounts

public LinearCounts()

Create a new instance of LinearCounts.

Method Detail

sortWordOrder

public void sortWordOrder(java.lang.String theText,
                          java.lang.String analysisTerm,
                          int minWordLength,
                          boolean wordStem)
                   throws java.lang.Exception

Sort all words in the text to determine the most popular ones. This version sorts a list of the top specified number of words and stores in a list of arrays of the form [word, count].

Parameters:

theText - the text to sort.

analysisTerm - specific analysis term to compare with. Can be null, or if not null then sets popularWordsNumber to 1.

minWordLength - minimum number of characters in a word.

wordStem - if true use word stemming to create the bag-of-words.

Throws:

java.lang.Exception - any error.

sortWordOrder

public void sortWordOrder(java.lang.String theText,
                          int popularWordsNumber,
                          int minWordLength,
                          boolean wordStem)
                   throws java.lang.Exception

Sort all words in the text to determine the most popular ones. This version sorts a list of the top specified number of words and stores in a list of arrays of the form [word, count].

Parameters:

theText - the text to sort.

popularWordsNumber - maximum number of popular words to store.

minWordLength - minimum number of characters in a word.

wordStem - if true use word stemming to create the bag-of-words.

Throws:

java.lang.Exception - any error.

sortWordOrder

protected void sortWordOrder(java.lang.String theText,
                             java.lang.String analysisTerm,
                             int popularWordsNumber,
                             int minWordLength,
                             boolean wordStem)
                      throws java.lang.Exception

Sort all words in the text to determine the most popular ones. This version sorts a list of the top specified number of words and stores in a list of arrays of the form [word, count].

Parameters:

theText - the text to sort.

analysisTerm - specific analysis term to compare with. Can be null, or if not null then sets popularWordsNumber to 1.

popularWordsNumber - maximum number of popular words to store.

minWordLength - minimum number of characters in a word.

wordStem - if true use word stemming to create the bag-of-words.

Throws:

java.lang.Exception - any error.

sortMostPopularSequences

public void sortMostPopularSequences(java.lang.String theText,
                                     java.lang.String analysisTerm,
                                     int minNestingSequenceNumber,
                                     int maxNestingSequenceNumber,
                                     int popularSequencesNumber,
                                     boolean wordStem)
                              throws java.lang.Exception

Sort all words in the text to determine the most popular sequences. This version sorts a list of the top specified number of sequences and stores in a list of arrays of the form [word, count].

Parameters:

theText - the text to sort.

analysisTerm - specific analysis term to compare with. Can be null, or if not null then sets popularWordsNumber to 1.

minNestingSequenceNumber - minimum number of words in a sequence.

maxNestingSequenceNumber - maximum number of words in a sequence.

popularSequencesNumber - number of sequences for each sequence length.

wordStem - if true use word stemming to create the bag-of-words.

Throws:

java.lang.Exception - any error.

sortMostPopularSequences

public void sortMostPopularSequences(java.util.ArrayList<java.lang.String> singleWordsList,
                                     java.lang.String analysisTerm,
                                     int minNestingSequenceNumber,
                                     int maxNestingSequenceNumber,
                                     int popularSequencesNumber,
                                     boolean wordStem)

Sort all words in the text to determine the most popular sequences. This version sorts a list of the top specified number of sequences and stores in a list of arrays of the form [word, count].

Parameters:

singleWordsList - single list of words to sort.

analysisTerm - specific analysis term to compare with. Can be null, or if not null then sets popularWordsNumber to 1.

minNestingSequenceNumber - minimum number of words in a sequence.

maxNestingSequenceNumber - maximum number of words in a sequence.

popularSequencesNumber - number of sequences for each sequence length.

wordStem - if true use word stemming to create the bag-of-words.

createBoW

protected void createBoW(java.lang.String theText,
                         boolean wordStem)
                  throws java.lang.Exception

Sort all words in the text to determine the most popular ones. This default implementation creates a bag-of-words structure with a word order based on count.

Parameters:

theText - the text to sort.

wordStem - if true use word stemming to create the bag-of-words. Default is false.

Throws:

java.lang.Exception - any error.

sequenceKey

protected java.lang.String sequenceKey(java.lang.String sequence,
                                       boolean wordStem)

Create a word sequence key, that might be different if word stemming is used. Also remove any commas, possibly included to separate terms.

Parameters:

sequence - the sequence to consider. Word tokens separated by spaces.

wordStem - true if word stem.

Returns:

a word sequence key, where stemming might truncate individual tokens.

getPopularWords

public java.util.ArrayList<java.lang.String[]> getPopularWords()

Get the popular words list.

Returns:

the value of popularWords.

getPopularWordSequences

public java.util.ArrayList<java.util.ArrayList<java.lang.String[]>> getPopularWordSequences()

Get the popular word sequences list.

Returns:

the value of popularSequences.