Evaluation Result

Similar to the dataset representation, the results are also represented in two classes RelationResult and the container DatasetResults.

RelationResult

RelationResult(relation_code: str, *, metadata: dict[str, Any], metric_values: Optional[dict[str, Any]] = None, instance_table: Optional[DataFrame] = None, answer_space: Optional[Series] = None, lazy_options: Optional[dict[str, Any]] = None, relation_info: Optional[dict[str, Any]] = None)

Methods:

Name	Description
activated	Return self or a copy of self with the instance_table loaded (lazy loading disabled).
from_path	Loads the evaluated relation from a JSONL file and associated metadata.
relation_info	Get or set additional relation information.
save	Save results to a file and export meta_data
search_path	Search path for instance files.

Attributes:

Name	Type	Description
answer_space	Series	The answer space of the relation.
instance_table	DataFrame	A pandas.DataFrame containing all items in the relation.
relation_code	str	The identifier of the relation.

answer_space property

answer_space: Series

The answer space of the relation.

instance_table property

instance_table: DataFrame

A pandas.DataFrame containing all items in the relation.

relation_code property

relation_code: str

The identifier of the relation.

activated

activated() -> Self

Return self or a copy of self with the instance_table loaded (lazy loading disabled).

from_path classmethod

from_path(path: PathLike, *, relation_code: Optional[str] = None, metadata: Optional[dict[str, Any]] = None, lazy: bool = True, fmt: InstanceTableFileFormat = None) -> RelationResult

Loads the evaluated relation from a JSONL file and associated metadata.

Parameters:

Name	Type	Description	Default
path	PathLike	The path to the relations instance table.	required

Returns:

Type	Description
RelationResult	An instance of the RelationResult class populated with data from the file.

Raises:

Type	Description
Exception	If there is an error in loading the file or processing the data.

relation_info

relation_info(**kw) -> dict[str, Any]

relation_info(key: str) -> Any

relation_info(key: Optional[str] = None, /, **kw: Any) -> Union[None, Any, dict[str, Any]]

Get or set additional relation information.

Use relation.relation_info(<field name>=<new value>) to set fields in the relation info dictionary. If a single field is selected, the respective value is returned. Otherwise the complete dictionary is returned.

Parameters:

Name	Type	Description	Default
key	Optional[str]	The field to retrieve.	None
**kw	Any	The fields to modify.	{}

Returns:

Type	Description
Union[None, Any, dict[str, Any]]	If a field is selected, the respective value is returned, otherwise, the complete info dictionary is
Union[None, Any, dict[str, Any]]	returned.

save

save(path: PathLike, fmt: InstanceTableFileFormat = None) -> Optional[Path]

Save results to a file and export meta_data

search_path classmethod

search_path(path: Path, relation_code: None = None, fmt: InstanceTableFileFormat = None) -> list[Path]

search_path(path: Path, relation_code: str, fmt: InstanceTableFileFormat = None) -> Path

search_path(path: Path, relation_code: Optional[str] = None, fmt: InstanceTableFileFormat = None) -> Union[list[Path], Path, None]

Search path for instance files.

DatasetResults

DatasetResults(results: Optional[list[RelationResult]] = None)

Container for relation results.

Methods:

Name	Description
from_path	Loads a results from a specified directory path.
get_metadata	Return metadata from the relations. If no keys are passed, all consistent values are returned.
get_metrics	Return the metrics for the relations in this dataset.

from_path classmethod

from_path(path: PathLike, *, lazy: bool = True, fmt: InstanceTableFileFormat = None, relation_info: Optional[PathLike] = None, **kwargs) -> DatasetResults

Loads a results from a specified directory path.

This method scans the directory for relation files and assembles them into a DatasetResults.

Parameters:

Name	Type	Description	Default
path	PathLike	The directory path where the dataset is stored.	required

Returns:

Type	Description
DatasetResults	An instance of DatasetResults loaded with the results from the directory.

Raises:

Type	Description
Exception	If there is an error in loading the dataset.

Usage

Loading all relation results for a dataset.

from results import DatasetResults
results = DatasetResults.load_from_path('/path/to/results/', dataset_name='BEAR')

get_metadata

get_metadata(key: None = None) -> dict[str, Any]

get_metadata(key: str) -> Any

get_metadata(key: list[str]) -> dict[str, Any]

get_metadata(key: Optional[Union[str, list[str]]] = None)

Return metadata from the relations. If no keys are passed, all consistent values are returned.

get_metrics

get_metrics(metrics: str, *, accumulate: Literal[True] = True, divide_support: bool = True) -> float

get_metrics(metrics: str, *, accumulate: Union[Literal[False], None, str], divide_support: bool = True) -> Series

get_metrics(metrics: Sequence[str], *, accumulate: Literal[True] = True, divide_support: bool = True) -> Series

get_metrics(metrics: Sequence[str], *, accumulate: Union[Literal[False], None, str], divide_support: bool = True) -> DataFrame

get_metrics(metrics: Union[str, Sequence[str]], *, accumulate: Union[bool, None, str] = True, divide_support: bool = True) -> Union[DataFrame, Series, float]

Return the metrics for the relations in this dataset.

Parameters:

Name	Type	Description	Default
accumulate	bool | str | None	Compute the metrics for groups of relations (e.g. over the domains) or compute the overall scores for the complete dataset by setting accumulate=True.	True
divide_support	bool	Set to true to divide the support (added by a relation to a group) by the number of groups it adds to (only relevant if there are multiple groups per relation i.e. when explode is set). This leads to a dataframe where the weightted mean is equal to the overall score.	True

Returns:

Type	Description
Union[DataFrame, Series, float]	pandas.DataFrame | pandas.Series: A Series or DataFrame with the selected metrics depending on whether all relations where accumulated.