A C D F G I N R S T V W

A

ArchiveRecordReader() - Constructor for class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

ArchiveRecordWritable - Class in io.archivesunleashed.io

Implements Hadoop Writable for Archive Records.

ArchiveRecordWritable() - Constructor for class io.archivesunleashed.io.ArchiveRecordWritable

Utility function.

ArchiveRecordWritable(ArchiveRecord) - Constructor for class io.archivesunleashed.io.ArchiveRecordWritable

Initialize Archive Record.

ArchiveRecordWritable.ArchiveFormat - Enum in io.archivesunleashed.io

Archive Formats that can be used.

ArcRecordUtils - Class in io.archivesunleashed.data

Utilities for working with ARCRecords (from archive.org APIs).

C

close() - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

createRecordReader(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacInputFormat

D

detectFormat() - Method in class io.archivesunleashed.io.ArchiveRecordWritable

Detect format of Archive Record.

F

fromBytes(byte[]) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Converts raw bytes into an ARCRecord.

fromBytes(byte[]) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Converts raw bytes into an WARCRecord.

G

getBodyContent(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Extracts contents of the body from an ARCRecord.

getBodyContent(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts contents of the body from a WARCRecord.

getContent(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Extracts raw contents from an ARCRecord (including HTTP headers).

getContent(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts raw contents from a WARCRecord (including HTTP headers).

getCurrentKey() - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

getCurrentValue() - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

getFormat() - Method in class io.archivesunleashed.io.ArchiveRecordWritable

Get format of Archive Record.

getProgress() - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

getRecord() - Method in class io.archivesunleashed.io.ArchiveRecordWritable

Get Archive Record.

getWarcResponseMimeType(byte[]) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Extracts the MIME type of WARC response records.

I

initialize(InputSplit, TaskAttemptContext) - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

io.archivesunleashed.data - package io.archivesunleashed.data

This package provides various data utilities for analyzing WARCs.

io.archivesunleashed.io - package io.archivesunleashed.io

This package provides various IO utilities for analyzing WARCs.

io.archivesunleashed.mapreduce - package io.archivesunleashed.mapreduce

This package provides various mapreduce utilities for analyzing WARCs.

isSplitable(JobContext, Path) - Method in class io.archivesunleashed.mapreduce.WacInputFormat

N

nextKeyValue() - Method in class io.archivesunleashed.mapreduce.WacInputFormat.ArchiveRecordReader

R

readFields(DataInput) - Method in class io.archivesunleashed.io.ArchiveRecordWritable

S

setFormat(ArchiveRecordWritable.ArchiveFormat) - Method in class io.archivesunleashed.io.ArchiveRecordWritable

Set format of Archive Record.

setRecord(ArchiveRecord) - Method in class io.archivesunleashed.io.ArchiveRecordWritable

Set Archive Record.

T

toBytes(ARCRecord) - Static method in class io.archivesunleashed.data.ArcRecordUtils

Converts ARC record into raw bytes.

toBytes(WARCRecord) - Static method in class io.archivesunleashed.data.WarcRecordUtils

Converts WARC record into raw bytes.

V

valueOf(String) - Static method in enum io.archivesunleashed.io.ArchiveRecordWritable.ArchiveFormat

Returns the enum constant of this type with the specified name.

values() - Static method in enum io.archivesunleashed.io.ArchiveRecordWritable.ArchiveFormat

Returns an array containing the constants of this enum type, in the order they are declared.

W

WacInputFormat - Class in io.archivesunleashed.mapreduce

Extends FileInputFormat for Web Archive Commons InputFormat.

WacInputFormat() - Constructor for class io.archivesunleashed.mapreduce.WacInputFormat

WacInputFormat.ArchiveRecordReader - Class in io.archivesunleashed.mapreduce

Extends RecordReader for Record Reader.

WarcRecordUtils - Class in io.archivesunleashed.data

Utilities for working with WARCRecords (from archive.org APIs).

write(DataOutput) - Method in class io.archivesunleashed.io.ArchiveRecordWritable

A C D F G I N R S T V W