c

io.archivesunleashed.app

CommandLineApp

class CommandLineApp extends AnyRef

Main application that parse command line arguments and invoke appropriate extractor.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CommandLineApp
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CommandLineApp(conf: CmdAppConf)

    conf

    Scallop option reader constructed with class CmdAppConf

Value Members

  1. def handler(): Any

    Prepare for invoking extractors.

    Prepare for invoking extractors.

    returns

    Any

  2. def process(): Any

    Process the handler.

    Process the handler.

    returns

    Any

  3. def saveCsv(d: Dataset[Row]): Unit

    Routine for saving Dataset obtained from querying DataFrames to CSV.

    Routine for saving Dataset obtained from querying DataFrames to CSV. Files may be merged according to options specified in 'partition' setting.

    d

    generic dataset obtained from querying DataFrame

    returns

    Unit

  4. def saveParquet(d: Dataset[Row]): Unit

    Routine for saving Dataset obtained from querying DataFrames to Parquet.

    Routine for saving Dataset obtained from querying DataFrames to Parquet. Files may be merged according to options specified in 'partition' setting.

    d

    generic dataset obtained from querying DataFrame

    returns

    Unit

  5. def setAppName(): String

    Set the app name.

    Set the app name.

    returns

    String

  6. def setSparkContext(sc: SparkContext): Unit

    Set Spark context to be used.

    Set Spark context to be used.

    sc

    either a brand new or existing Spark context

  7. def verifyArgumentsOrExit(): Unit

    Verify the validity of command line arguments regarding input and output files.

    Verify the validity of command line arguments regarding input and output files.

    All input files need to exist, and ouput files should not exist, for this to pass. Throws exception if condition is not met.

    returns

    Unit

    Exceptions thrown

    IllegalArgumentException exception thrown