Bases: ABC
This abstract class serves as a blueprint for creating specific validator classes
that handle the validation process of machine learning models using PyTorch.
It defines the essential components and methods that any concrete validator class
must implement, namely the overall validation loop, while the per-batch validation
logic is delegated to an AbstractValidationStep instance passed at construction
time. This allows the same validator class to be reused simply by changing the
validation step implementation.
Notes
This can be used also in distributed validation scenarios, but the implementation
of the validate method should take into account the distributed setting.
We suggest to check the example :doc:examples/mnist_training.py for a reference
implementation of a validator that can be used in distributed validation.
| Parameters: |
-
validation_step
(AbstractValidationStep)
–
The validation step used to evaluate the model's performance on a single batch of data.
-
device
(str or device)
–
The device (e.g., 'cpu' or 'cuda:0') on which the model and data will
be loaded for validation.
|
Source code in aidevelopementtoolkit/torch_utils/AbstractValidator.py
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138 | class AbstractValidator(ABC):
"""This abstract class serves as a blueprint for creating specific validator classes
that handle the validation process of machine learning models using PyTorch.
It defines the essential components and methods that any concrete validator class
must implement, namely the overall validation loop, while the per-batch validation
logic is delegated to an `AbstractValidationStep` instance passed at construction
time. This allows the same validator class to be reused simply by changing the
validation step implementation.
Notes
-----
This can be used also in distributed validation scenarios, but the implementation
of the `validate` method should take into account the distributed setting.
We suggest to check the example :doc:`examples/mnist_training.py` for a reference
implementation of a validator that can be used in distributed validation.
Parameters
----------
validation_step : AbstractValidationStep
The validation step used to evaluate the model's performance on a single batch of data.
device : str or torch.device
The device (e.g., 'cpu' or 'cuda:0') on which the model and data will
be loaded for validation.
"""
def __init__(
self,
validation_step: AbstractValidationStep,
device: Union[str, torch.device],
) -> None:
self.validation_step = validation_step
self.device = torch.device(device) if isinstance(device, str) else device
@abstractmethod
def validate(
self,
model: nn.Module,
data_loader: torch.utils.data.DataLoader,
current_epoch: int,
**kwargs: Any,
) -> Any:
"""Validate the model on the given data loader.
Parameters
----------
model : nn.Module
The model to be validated.
data_loader : torch.utils.data.DataLoader
The data loader providing the validation data.
current_epoch : int
The current epoch number, which can be used for logging or checkpointing purposes.
**kwargs : Dict[str, Any]
Additional keyword arguments that subclasses can use to receive extra
information needed for a specific validation loop implementation.
Returns
-------
Any
The result of the validation, which can be any type depending on the
specific implementation in the subclass.
"""
logger.error("Subclasses must implement this method.", NotImplementedError)
|
Validate the model on the given data loader.
| Parameters: |
-
model
(Module)
–
The model to be validated.
-
data_loader
(DataLoader)
–
The data loader providing the validation data.
-
current_epoch
(int)
–
The current epoch number, which can be used for logging or checkpointing purposes.
-
**kwargs
(Dict[str, Any], default:
{}
)
–
Additional keyword arguments that subclasses can use to receive extra
information needed for a specific validation loop implementation.
|
| Returns: |
-
Any
–
The result of the validation, which can be any type depending on the
specific implementation in the subclass.
|
Source code in aidevelopementtoolkit/torch_utils/AbstractValidator.py
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138 | @abstractmethod
def validate(
self,
model: nn.Module,
data_loader: torch.utils.data.DataLoader,
current_epoch: int,
**kwargs: Any,
) -> Any:
"""Validate the model on the given data loader.
Parameters
----------
model : nn.Module
The model to be validated.
data_loader : torch.utils.data.DataLoader
The data loader providing the validation data.
current_epoch : int
The current epoch number, which can be used for logging or checkpointing purposes.
**kwargs : Dict[str, Any]
Additional keyword arguments that subclasses can use to receive extra
information needed for a specific validation loop implementation.
Returns
-------
Any
The result of the validation, which can be any type depending on the
specific implementation in the subclass.
"""
logger.error("Subclasses must implement this method.", NotImplementedError)
|