cnlpt.CnlpModelForClassification module

Module containing the CNLP transformer model.

class cnlpt.CnlpModelForClassification.ClassificationHead

Bases: Module

__init__(config, num_labels, hidden_size=- 1)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(features, *kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cnlpt.CnlpModelForClassification.RepresentationProjectionLayer

Bases: Module

__init__(config, layer=10, tokens=False, tagger=False, relations=False, num_attention_heads=- 1, head_size=64)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(features, event_tokens, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cnlpt.CnlpModelForClassification.CnlpConfig

Bases: PretrainedConfig

The config class for CnlpModelForClassification.

Parameters

  • encoder_name – the encoder name to use with transformers.AutoConfig.from_pretrained()

  • finetuning_task (Optional[str]) – TODO define

  • num_labels_list (List[int]) – the number of labels for each task

  • layer (int) – the index of the encoder layer to extract features from

  • tokens (bool) – TODO define

  • num_rel_attention_heads (int) – the number of features/attention heads to use in the NxN relation classifier

  • rel_attention_head_dims (int) – the number of parameters in each attention head in the NxN relation classifier

  • tagger (List[bool]) – for each task, whether the task is a sequence tagging task

  • relations (List[bool]) – for each task, whether the task is a relation extraction task

  • use_prior_tasks (bool) – whether to use the outputs from the previous tasks as additional inputs for subsequent tasks

  • num_tokens (int) – TODO define

  • **kwargs – arguments for transformers.PretrainedConfig

__init__(encoder_name='roberta-base', finetuning_task=None, num_labels_list=[], layer=- 1, tokens=False, num_rel_attention_heads=12, rel_attention_head_dims=64, tagger=[False], relations=[False], use_prior_tasks=False, num_tokens=- 1, **kwargs)
class cnlpt.CnlpModelForClassification.CnlpModelForClassification

Bases: PreTrainedModel

The CNLP transformer model.

Parameters

  • class_weights (Optional[List[float]]) – if provided, the weights to use for each task when computing the loss

  • final_task_weight (float) – the weight to use for the final task when computing the loss; default 1.0.

  • argument_regularization (float) – if provided, the argument regularization to use when computing the loss

  • freeze (bool) – whether to freeze the weights of the encoder

  • bias_fit (bool) – whether to fine-tune only the bias of the encoder

config_class

alias of CnlpConfig

__init__(config, *, class_weights=None, final_task_weight=1.0, argument_regularization=- 1, freeze=False, bias_fit=False)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

compute_loss(task_logits, labels, task_ind, task_num_labels, batch_size, seq_len, state)

Computes the loss for a single batch and a single task.

Parameters

  • task_logits –

  • labels –

  • task_ind –

  • task_num_labels –

  • batch_size –

  • seq_len –

  • state –

apply_arg_reg(logits, attention_mask, state)

Applies argument regularization to the logits for a single batch on all tasks.

Parameters

  • logits (List[torch.FloatTensor]) – the computed logits of the batch.

  • attention_mask (torch.LongTensor) – the attention mask for the batch.

  • state – the state dict containing the loss for the batch.

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, event_tokens=None)

Forward method.

Parameters

  • input_ids (torch.LongTensor of shape (batch_size, sequence_len), optional) – A batch of chunked documents as tokenizer indices.

  • attention_mask (torch.LongTensor of shape (batch_size, sequence_len), optional) – Attention masks for the batch.

  • token_type_ids (torch.LongTensor of shape (batch_size, sequence_len), optional) – Token type IDs for the batch.

  • position_ids – (torch.LongTensor of shape (batch_size, sequence_len), optional): Position IDs for the batch.

  • head_mask (torch.LongTensor of shape (num_heads,) or (num_layers, num_heads), optional) – Token encoder head mask.

  • inputs_embeds (torch.FloatTensor of shape (batch_size, sequence_len, hidden_size), optional) – A batch of chunked documents as token embeddings.

  • labels (torch.LongTensor of shape (batch_size,), optional) – Labels for computing the sequence classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If config.num_labels > 1 a classification loss is computed (Cross-Entropy).

  • output_attentions (bool, optional) – Whether or not to return the attentions tensors of all attention layers.

  • output_hidden_states – not used.

  • event_tokens – TODO define

Returns: (transformers.SequenceClassifierOutput) the output of the model