A C D F G I N R S T V W

A

ArcRecordReader() - Constructor for class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

ArcRecordUtils - Class in io.archivesunleashed.data

Utilities for working with ARCRecords (from archive.org APIs).

ArcRecordWritable - Class in io.archivesunleashed.io

Implements Hadoop Writable for ARC Records.

ArcRecordWritable() - Constructor for class io.archivesunleashed.io.ArcRecordWritable

Utility function.

ArcRecordWritable(ARCRecord) - Constructor for class io.archivesunleashed.io.ArcRecordWritable

Initialize ARC Record.

C

close() - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

close() - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

close() - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

createRecordReader(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat

createRecordReader(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat

createRecordReader(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat

D

detectFormat() - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

Detect format of Generic Archive Record.

F

fromBytes(byte[]) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Converts raw bytes into an ARCRecord.

fromBytes(byte[]) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Converts raw bytes into an WARCRecord.

G

GenericArchiveRecordReader() - Constructor for class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

GenericArchiveRecordWritable - Class in io.archivesunleashed.io

Implements Hadoop Writable for Generic Archive Records.

GenericArchiveRecordWritable() - Constructor for class io.archivesunleashed.io.GenericArchiveRecordWritable

Utility function.

GenericArchiveRecordWritable(ArchiveRecord) - Constructor for class io.archivesunleashed.io.GenericArchiveRecordWritable

Initialize Generic Archive Record.

GenericArchiveRecordWritable.ArchiveFormat - Enum in io.archivesunleashed.io

Archive Formats that can be used.

getBodyContent(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Extracts contents of the body from an ARCRecord.

getBodyContent(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts contents of the body from a WARCRecord.

getContent(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Extracts raw contents from an ARCRecord (including HTTP headers).

getContent(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts raw contents from a WARCRecord (including HTTP headers).

getCurrentKey() - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

getCurrentKey() - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

getCurrentKey() - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

getCurrentValue() - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

getCurrentValue() - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

getCurrentValue() - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

getFormat() - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

Get format of Generic Archive Record.

getProgress() - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

getProgress() - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

getProgress() - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

getRecord() - Method in class io.archivesunleashed.io.ArcRecordWritable

Get ARC Record.

getRecord() - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

Get Generic Archive Record.

getRecord() - Method in class io.archivesunleashed.io.WarcRecordWritable

Get WARC Record.

getWarcResponseMimeType(byte[]) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts the MIME type of WARC response records.

I

initialize(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

initialize(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

initialize(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

io.archivesunleashed.data - package io.archivesunleashed.data

This package provides various data utilities for analyzing WARCs.

io.archivesunleashed.io - package io.archivesunleashed.io

This package provides various IO utilities for analyzing WARCs.

io.archivesunleashed.mapreduce - package io.archivesunleashed.mapreduce

This package provides various mapreduce utilities for analyzing WARCs.

isSplitable(JobContext, Path) - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat

isSplitable(JobContext, Path) - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat

isSplitable(JobContext, Path) - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat

N

nextKeyValue() - Method in class io.archivesunleashed.mapreduce.WacArcInputFormat.ArcRecordReader

nextKeyValue() - Method in class io.archivesunleashed.mapreduce.WacGenericInputFormat.GenericArchiveRecordReader

nextKeyValue() - Method in class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

R

readFields(DataInput) - Method in class io.archivesunleashed.io.ArcRecordWritable

readFields(DataInput) - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

readFields(DataInput) - Method in class io.archivesunleashed.io.WarcRecordWritable

S

setFormat(GenericArchiveRecordWritable.ArchiveFormat) - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

Set format of Generic Archive Record.

setRecord(ARCRecord) - Method in class io.archivesunleashed.io.ArcRecordWritable

Set ARC Record.

setRecord(ArchiveRecord) - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

Set Generic Archive Record.

setRecord(WARCRecord) - Method in class io.archivesunleashed.io.WarcRecordWritable

Set WARC Record.

T

toBytes(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Converts ARC record into raw bytes.

toBytes(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Converts WARC record into raw bytes.

V

valueOf(String) - Static method in enum io.archivesunleashed.io.GenericArchiveRecordWritable.ArchiveFormat

Returns the enum constant of this type with the specified name.

values() - Static method in enum io.archivesunleashed.io.GenericArchiveRecordWritable.ArchiveFormat

Returns an array containing the constants of this enum type, in the order they are declared.

W

WacArcInputFormat - Class in io.archivesunleashed.mapreduce

Extends FileInputFormat for Web Archive Commons ARC InputFormat.

WacArcInputFormat() - Constructor for class io.archivesunleashed.mapreduce.WacArcInputFormat

WacArcInputFormat.ArcRecordReader - Class in io.archivesunleashed.mapreduce

Extends RecordReader for ARC Record Reader.

WacGenericInputFormat - Class in io.archivesunleashed.mapreduce

Extends FileInputFormat for Web Archive Commons Generic InputFormat.

WacGenericInputFormat() - Constructor for class io.archivesunleashed.mapreduce.WacGenericInputFormat

WacGenericInputFormat.GenericArchiveRecordReader - Class in io.archivesunleashed.mapreduce

Extends RecordReader for Generic Record Reader.

WacWarcInputFormat - Class in io.archivesunleashed.mapreduce

Extends FileInputFormat for Web Archive Commons WARC InputFormat.

WacWarcInputFormat() - Constructor for class io.archivesunleashed.mapreduce.WacWarcInputFormat

WacWarcInputFormat.WarcRecordReader - Class in io.archivesunleashed.mapreduce

Extends RecordReader for WARC Record Reader.

WarcRecordReader() - Constructor for class io.archivesunleashed.mapreduce.WacWarcInputFormat.WarcRecordReader

WarcRecordUtils - Class in io.archivesunleashed.data

Utilities for working with WARCRecords (from archive.org APIs).

WarcRecordWritable - Class in io.archivesunleashed.io

Implements Hadoop Writable for WARC Records.

WarcRecordWritable() - Constructor for class io.archivesunleashed.io.WarcRecordWritable

Utility function.

WarcRecordWritable(WARCRecord) - Constructor for class io.archivesunleashed.io.WarcRecordWritable

Initialize WARC Record.

write(DataOutput) - Method in class io.archivesunleashed.io.ArcRecordWritable

write(DataOutput) - Method in class io.archivesunleashed.io.GenericArchiveRecordWritable

write(DataOutput) - Method in class io.archivesunleashed.io.WarcRecordWritable

A C D F G I N R S T V W