net.librec.data.convertor

Class TextDataConvertor

java.lang.Object

net.librec.job.progress.ProgressReporter

net.librec.data.convertor.AbstractDataConvertor

net.librec.data.convertor.TextDataConvertor

All Implemented Interfaces:

java.lang.Runnable, DataConvertor, Progressable

public class TextDataConvertor
extends AbstractDataConvertor

A TextDataConvertor is a class to convert a data file from CSV format to a target format.

Author:

WangYuFeng and liuxz

Field Summary

Fields inherited from class net.librec.data.convertor.AbstractDataConvertor

datetimeMatrix, preferenceMatrix, sparseTensor

Fields inherited from class net.librec.job.progress.ProgressReporter

PROGRESS_INTERVAL

Constructor Summary

Constructors

Constructor and Description
TextDataConvertor(java.lang.String inputDataPath) Initializes a newly created TextDataConvertor object with the path of the input data file.
TextDataConvertor(java.lang.String dataColumnFormat, java.lang.String inputDataPath) Initializes a newly created TextDataConvertor object with the path and format of the input data file.
TextDataConvertor(java.lang.String dataColumnFormat, java.lang.String inputDataPath, double binThold) Initializes a newly created TextDataConvertor object with the path and format of the input data file.
TextDataConvertor(java.lang.String dataColumnFormat, java.lang.String inputDataPath, double binThold, com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> userIds, com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> itemIds) Initializes a newly created TextDataConvertor object with the path and format of the input data file.

Method Summary

Methods

Modifier and Type	Method and Description
double	getDataFileRate() Return rate of alreadyLoaded/allData in one file.
double	getFilePathRate() Return rate of loading files in data directory.
int	getItemId(java.lang.String rawId) Return an item's inner id by its raw id.
com.google.common.collect.BiMap<java.lang.String,java.lang.Integer>	getItemIds() Return item {rawid, inner id} mappings
double	getLoadAllFileRate() Return rate of alreadyLoaded/allData in all files.
int	getUserId(java.lang.String rawId) Return a user's inner id by his raw id.
com.google.common.collect.BiMap<java.lang.String,java.lang.Integer>	getUserIds() Return user {rawid, inner id} mappings
int	numItems() Return the number of items.
int	numUsers() Return the number of users.
void	processData() Process the input data.
void	progress() Set the progress for job status.
void	setTimeUnit(java.util.concurrent.TimeUnit timeUnit) Set the time unit of the data file.

Methods inherited from class net.librec.data.convertor.AbstractDataConvertor

getDatetimeMatrix, getPreferenceMatrix, getSparseTensor

Methods inherited from class net.librec.job.progress.ProgressReporter

getJobStatus, progressx, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextDataConvertor

public TextDataConvertor(java.lang.String inputDataPath)

Initializes a newly created TextDataConvertor object with the path of the input data file.

Parameters:

inputDataPath - the path of the input data file

TextDataConvertor

public TextDataConvertor(java.lang.String dataColumnFormat,
                 java.lang.String inputDataPath)

Initializes a newly created TextDataConvertor object with the path and format of the input data file.

Parameters:

dataColumnFormat - the path of the input data file

inputDataPath - the format of the input data file

TextDataConvertor

public TextDataConvertor(java.lang.String dataColumnFormat,
                 java.lang.String inputDataPath,
                 double binThold)

Initializes a newly created TextDataConvertor object with the path and format of the input data file.

Parameters:

dataColumnFormat - the path of the input data file

inputDataPath - the format of the input data file

binThold - the threshold to binarize a rating. If a rating is greater than the threshold, the value will be 1; otherwise 0. To disable this appender, i.e., keep the original rating value, set the threshold a negative value

TextDataConvertor

public TextDataConvertor(java.lang.String dataColumnFormat,
                 java.lang.String inputDataPath,
                 double binThold,
                 com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> userIds,
                 com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> itemIds)

Initializes a newly created TextDataConvertor object with the path and format of the input data file.

Parameters:

dataColumnFormat - the path of the input data file

inputDataPath - the format of the input data file

binThold - the threshold to binarize a rating. If a rating is greater than the threshold, the value will be 1; otherwise 0. To disable this appender, i.e., keep the original rating value, set the threshold a negative value

userIds - userId to userIndex map

itemIds - itemId to itemIndex map

Method Detail

processData

public void processData()
                 throws java.io.IOException

Process the input data.

Throws:

java.io.IOException - if the inputDataPath is not valid.

progress

public void progress()

Set the progress for job status.

getFilePathRate

public double getFilePathRate()

Return rate of loading files in data directory.

Returns:

loadFilePathRate

getDataFileRate

public double getDataFileRate()

Return rate of alreadyLoaded/allData in one file.

Returns:

loadDataFileRate

getLoadAllFileRate

public double getLoadAllFileRate()

Return rate of alreadyLoaded/allData in all files.

Returns:

loadAllFileRate

numUsers

public int numUsers()

Return the number of users.

Returns:

number of users

numItems

public int numItems()

Return the number of items.

Returns:

number of items

getUserId

public int getUserId(java.lang.String rawId)

Return a user's inner id by his raw id.

Parameters:

rawId - raw user id as String

Returns:

inner user id as int

getItemId

public int getItemId(java.lang.String rawId)

Return an item's inner id by its raw id.

Parameters:

rawId - raw item id as String

Returns:

inner item id as int

getUserIds

public com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> getUserIds()

Return user {rawid, inner id} mappings

Returns:

userIds

getItemIds

public com.google.common.collect.BiMap<java.lang.String,java.lang.Integer> getItemIds()

Return item {rawid, inner id} mappings

Returns:

itemIds

setTimeUnit

public void setTimeUnit(java.util.concurrent.TimeUnit timeUnit)

Set the time unit of the data file.

Parameters:

timeUnit - the time unit to be set for the data file