org.ai_heuristic.model

Class BagOfWords

java.lang.Object

org.ai_heuristic.model.BagOfWords

Direct Known Subclasses:

MetaBagOfWords

public class BagOfWords
extends java.lang.Object

This class can be used to create a bag-of-words list from a text document.

Author:

Kieran Greer

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.HashMap<java.lang.String,java.lang.Integer>	bagOfWords This stores the bag of words.
protected java.lang.String	name Unique name or ID
protected java.util.ArrayList<java.lang.String>	wordOrder Word ordering from most to least counts

Constructor Summary

Constructors

Constructor and Description
BagOfWords() Create a new instance of BagOfWords.

Method Summary

Modifier and Type	Method and Description
void	createBagOfWords(java.lang.String theText) Parse the text that is entered to create the bag of words.
void	createBagOfWords(java.lang.String theText, boolean wordStem) Parse the text that is entered to create the bag of words.
BagOfWords	difference(BagOfWords thisBagOfWords) Create a new bag of words that is the difference between of this one and the one passed in.
float	dotproduct(BagOfWords thisBagOfWords) Calculate the dotproduct value of this bag-of-words with the one passed in.
java.util.HashMap<java.lang.String,java.lang.Integer>	getBagOfWords() Get a copy of the bag of words structure.
java.lang.String	getName() Get the name or id of this bag-of-words.
int	getTotalWordCount() Get the total count of all instances of each word.
java.util.ArrayList<java.lang.String>	getWordOrder() Get the ordered word list from most to least counts.
BagOfWords	intersection(BagOfWords thisBagOfWords) Create a new bag of words that is the intersection of this one with the one passed in.
float	magnitude(BagOfWords thisBagOfWords) Calculate the magnitude value of this bag-of-words with the one passed in.
protected void	orderBagOfWords() Create a ordered list of most to least counts for the bag of words.
void	removeWords(java.util.ArrayList<java.lang.String> toRemove) Remove the list of words from the BOW structures - bagOfWords and wordOrder.
boolean	sameWordList(BagOfWords compareWith) Return true if this bag-of-words has the same word list as the bag-of-words passed in.
void	setBagOfWords(java.util.HashMap<java.lang.String,java.lang.Integer> thisBagOfWords) Set the bag of words for this class to process.
void	setBagOfWords(java.util.HashMap<java.lang.String,java.lang.Integer> thisBagOfWords, java.util.ArrayList<java.lang.String> thisWordOrder) Set the bag of words for this class to process.
void	setName(java.lang.String thisName) Set the name or id of this bag-of-words.
BagOfWords	subtract(BagOfWords thisBagOfWords) Create a new bag of words that is the subtraction of the bag passed in from this one.
org.licas_xml.abs.Element	toXml() Convert this bag of words into an XML format.
BagOfWords	union(BagOfWords thisBagOfWords) Create a new bag of words that is the union of this one with the one passed in.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

name

protected java.lang.String name

Unique name or ID

bagOfWords

protected java.util.HashMap<java.lang.String,java.lang.Integer> bagOfWords

This stores the bag of words. Key is the word while value is the word occurrence count.

wordOrder

protected java.util.ArrayList<java.lang.String> wordOrder

Word ordering from most to least counts

Constructor Detail

BagOfWords

public BagOfWords()

Create a new instance of BagOfWords.

Method Detail

createBagOfWords

public void createBagOfWords(java.lang.String theText)
                      throws java.lang.Exception

Parse the text that is entered to create the bag of words. This option does not apply word stemming, to combine the same words with different endings.

Parameters:

theText - the text sequence to parse.

Throws:

java.lang.Exception - any error.

createBagOfWords

public void createBagOfWords(java.lang.String theText,
                             boolean wordStem)
                      throws java.lang.Exception

Parse the text that is entered to create the bag of words. This option allows for word stemming, to combine the same words with different endings.

Parameters:

theText - the text sequence to parse.

wordStem - if true re-parse using word stemming, if false leave as original words.

Throws:

java.lang.Exception - any error.

orderBagOfWords

protected void orderBagOfWords()
                        throws java.lang.Exception

Create a ordered list of most to least counts for the bag of words.

Throws:

java.lang.Exception - any error.

removeWords

public void removeWords(java.util.ArrayList<java.lang.String> toRemove)

Remove the list of words from the BOW structures - bagOfWords and wordOrder.

Parameters:

toRemove - list of words to remove.

sameWordList

public boolean sameWordList(BagOfWords compareWith)

Return true if this bag-of-words has the same word list as the bag-of-words passed in.

Parameters:

compareWith - the bag-of-words to compare with.

Returns:

true if comparison is true.

subtract

public BagOfWords subtract(BagOfWords thisBagOfWords)
                    throws java.lang.Exception

Create a new bag of words that is the subtraction of the bag passed in from this one. The counts for items in the bag passed in are subtracted from the counts of similar items in this bag. If the result is less than or equal to 0, then the item is not kept. Any other items in this bag only are also kept.

Parameters:

thisBagOfWords - the bag of words to intersect with.

Returns:

a new BagOfWords structure with the appropriate word counts.

Throws:

java.lang.Exception - any error.

difference

public BagOfWords difference(BagOfWords thisBagOfWords)
                      throws java.lang.Exception

Create a new bag of words that is the difference between of this one and the one passed in. This should be the reciprocal of the intersection. The counts for items in the bag passed in are subtracted from the counts of similar items in this bag and the absolute value taken. Any other items in either bag are also kept.

Parameters:

thisBagOfWords - the bag of words to intersect with.

Returns:

a new BagOfWords structure with the appropriate word counts.

Throws:

java.lang.Exception - any error.

intersection

public BagOfWords intersection(BagOfWords thisBagOfWords)
                        throws java.lang.Exception

Create a new bag of words that is the intersection of this one with the one passed in. The counts for the same item are summed.

Parameters:

thisBagOfWords - the bag of words to intersect with.

Returns:

a new BagOfWords structure with the appropriate word counts.

Throws:

java.lang.Exception - any error.

union

public BagOfWords union(BagOfWords thisBagOfWords)
                 throws java.lang.Exception

Create a new bag of words that is the union of this one with the one passed in. The counts for the same item in both bags are summed, or the item is added as is. This is the same as the addition operation.

Parameters:

thisBagOfWords - the bag of words to combine with.

Returns:

a new BagOfWords structure with the appropriate word counts.

Throws:

java.lang.Exception - any error.

dotproduct

public float dotproduct(BagOfWords thisBagOfWords)

Calculate the dotproduct value of this bag-of-words with the one passed in. The counts for the same item in both bags are multiplied and summed.

Parameters:

thisBagOfWords - the bag of words to combine with.

Returns:

the total dotproduct count.

magnitude

public float magnitude(BagOfWords thisBagOfWords)

Calculate the magnitude value of this bag-of-words with the one passed in. The counts for each item in a bag is squared and summed. Both counts are then multiplied together.

Parameters:

thisBagOfWords - the bag of words to combine with.

Returns:

the total magnitude count.

getTotalWordCount

public int getTotalWordCount()

Get the total count of all instances of each word.

Returns:

the total count of all counts for all words.

setName

public void setName(java.lang.String thisName)

Set the name or id of this bag-of-words.

Parameters:

thisName - the bag-of-words name.

getName

public java.lang.String getName()

Get the name or id of this bag-of-words.

Returns:

the value of name.

setBagOfWords

public void setBagOfWords(java.util.HashMap<java.lang.String,java.lang.Integer> thisBagOfWords)
                   throws java.lang.Exception

Set the bag of words for this class to process. This also orders the bag of words.

Parameters:

thisBagOfWords - the bag of words structure.

Throws:

java.lang.Exception - any error.

setBagOfWords

public void setBagOfWords(java.util.HashMap<java.lang.String,java.lang.Integer> thisBagOfWords,
                          java.util.ArrayList<java.lang.String> thisWordOrder)
                   throws java.lang.Exception

Set the bag of words for this class to process. This also orders the bag of words.

Parameters:

thisBagOfWords - the bag of words structure.

thisWordOrder - word ordering for the bag-of-words.

Throws:

java.lang.Exception - any error.

getBagOfWords

public java.util.HashMap<java.lang.String,java.lang.Integer> getBagOfWords()

Get a copy of the bag of words structure. This does not return a direct reference because the bow has a separate word ordering structure. If changes need to be made, then use the 'set' method to update.

Returns:

the value of bagOfWords.

getWordOrder

public java.util.ArrayList<java.lang.String> getWordOrder()

Get the ordered word list from most to least counts. This does not return a direct reference because the word ordering relates directly to the bow.

Returns:

the value of wordOrder.

toXml

public org.licas_xml.abs.Element toXml()
                                throws java.lang.Exception

Convert this bag of words into an XML format.

Returns:

the xml-based representation of this bag of words.

Throws:

java.lang.Exception - any error.