Package opennlp.uima.tokenize

# Class Tokenizer

java.lang.Object
org.apache.uima.analysis_component.AnalysisComponent_ImplBase
org.apache.uima.analysis_component.Annotator_ImplBase
org.apache.uima.analysis_component.CasAnnotator_ImplBase
opennlp.uima.tokenize.AbstractTokenizer
opennlp.uima.tokenize.Tokenizer

All Implemented Interfaces:
  `org.apache.uima.analysis_component.AnalysisComponent`

public final class Tokenizer extends AbstractTokenizer
An OpenNLP-based Tokenizer annotator.

Mandatory parameters:

Type | Name | Description
--- | --- | ---
String | `opennlp.uima.ModelName` | The name of the model file
String | `opennlp.uima.SentenceType` | The full name of the sentence type
String | `opennlp.uima.TokenType` | The full name of the token type

Optional parameters:

Type | Name | Description
--- | --- | ---
String | `opennlp.uima.ProbabilityFeature` | The name of the double probability feature (not set by default)

See Also:
  - `TokenizerME`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `Tokenizer()`
  Initializes a new instance.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `destroy()`
  Releases allocated resources.
  `void`
  `initialize(org.apache.uima.UimaContext context)`
  Initializes the current instance with the given context.
  `void`
  `typeSystemInit(org.apache.uima.cas.TypeSystem typeSystem)`
  Initializes the `type system`.

  ### Methods inherited from class opennlp.uima.tokenize.AbstractTokenizer

  `process`

  ### Methods inherited from class org.apache.uima.analysis_component.CasAnnotator_ImplBase

  `getRequiredCasInterface, process`

  ### Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

  `getCasInstancesRequired, hasNext, next`

  ### Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

  `batchProcessComplete, collectionProcessComplete, reconfigure, setResultSpecification`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### Tokenizer

    public Tokenizer()
    Initializes a new instance.

    Note: Use `initialize(UimaContext)` to initialize this instance. Not use the constructor.

- ## Method Details

  - ### initialize

    public void initialize(org.apache.uima.UimaContext context) throws org.apache.uima.resource.ResourceInitializationException
    Initializes the current instance with the given context.

    Note: Do all initialization in this method, do not use the constructor.

    Specified by:
      `initialize` in interface `org.apache.uima.analysis_component.AnalysisComponent`
    Overrides:
      `initialize` in class `AbstractTokenizer`
    Throws:
      `org.apache.uima.resource.ResourceInitializationException`

  - ### typeSystemInit

    public void typeSystemInit(org.apache.uima.cas.TypeSystem typeSystem) throws org.apache.uima.analysis_engine.AnalysisEngineProcessException
    Initializes the `type system`.
    Overrides:
      `typeSystemInit` in class `AbstractTokenizer`
    Throws:
      `org.apache.uima.analysis_engine.AnalysisEngineProcessException`

  - ### destroy

    public void destroy()
    Releases allocated resources.
    Specified by:
      `destroy` in interface `org.apache.uima.analysis_component.AnalysisComponent`
    Overrides:
      `destroy` in class `org.apache.uima.analysis_component.AnalysisComponent_ImplBase`