Evaluation¶

Contained within this file are experimental interfaces for working with the Synapse Python Client. Unless otherwise noted these interfaces are subject to change at any time. Use at your own risk.

API reference¶

synapseclient.models.Evaluation `dataclass` ¶

Bases: EvaluationSynchronousProtocol

An Evaluation is the core object of the Evaluation API, used to support collaborative data analysis challenges in Synapse.

An Evaluation object represents an evaluation queue in Synapse: https://rest-docs.synapse.org/rest/org/sagebionetworks/evaluation/model/Evaluation.html

ATTRIBUTE	DESCRIPTION
`id`	The unique immutable ID for this Evaluation. TYPE: `Optional[str]`
`etag`	Synapse employs an Optimistic Concurrency Control (OCC) scheme to handle concurrent updates. The eTag changes every time an Evaluation is updated; it is used to detect when a client's copy of an Evaluation is out-of-date. TYPE: `Optional[str]`
`name`	The name of this Evaluation. TYPE: `Optional[str]`
`description`	A text description of this Evaluation. TYPE: `Optional[str]`
`owner_id`	The ID of the Synapse user who created this Evaluation. TYPE: `Optional[str]`
`created_on`	The date on which Evaluation was created. TYPE: `Optional[str]`
`content_source`	The Synapse ID of the Entity to which this Evaluation belongs, e.g. a reference to a Synapse project. TYPE: `Optional[str]`
`submission_instructions_message`	Message to display to users detailing acceptable formatting for Submissions to this Evaluation. TYPE: `Optional[str]`
`submission_receipt_message`	Message to display to users upon successful submission to this Evaluation. TYPE: `Optional[str]`

Create a new evaluation in a project

Create a new evaluation on Synapse by storing an evaluation object with the required fields. If there are any fields missing, an error will be raised.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(
    name="My Challenge Evaluation",
    description="Evaluation for my data challenge",
    content_source="syn123456",
    submission_instructions_message="Submit CSV files only",
    submission_receipt_message="Thank you for your submission!",
)
created = evaluation.store()

Update an existing evaluation retrieved from Synapse by ID

You can use the store method to create and update evaluations.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
evaluation.description = "Updated description for my evaluation"
updated = evaluation.store()

Retrieve and update the ACL of an evaluation

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
acl = evaluation.get_acl()

# Let's grant this team READ and SUBMIT permissions
team_id = "123456"
evaluation.update_acl(principal_id=team_id, access_type=["READ", "SUBMIT"])

# Now let's revoke all permissions from this team
team_to_revoke_from = "654321"
evaluation.update_acl(principal_id=team_to_revoke_from, access_type=[])

Delete an evaluation

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
evaluation.delete()

Source code in synapseclient/models/evaluation.py

@dataclass
@async_to_sync
class Evaluation(EvaluationSynchronousProtocol):
    """
    An Evaluation is the core object of the Evaluation API, used to support collaborative data analysis challenges in Synapse.

    An `Evaluation` object represents an evaluation queue in Synapse:
    <https://rest-docs.synapse.org/rest/org/sagebionetworks/evaluation/model/Evaluation.html>

    Attributes:
        id: The unique immutable ID for this Evaluation.
        etag: Synapse employs an Optimistic Concurrency Control (OCC) scheme to handle concurrent updates.
              The eTag changes every time an Evaluation is updated; it is used to detect when a client's copy
              of an Evaluation is out-of-date.
        name: The name of this Evaluation.
        description: A text description of this Evaluation.
        owner_id: The ID of the Synapse user who created this Evaluation.
        created_on: The date on which Evaluation was created.
        content_source: The Synapse ID of the Entity to which this Evaluation belongs,
                        e.g. a reference to a Synapse project.
        submission_instructions_message: Message to display to users detailing acceptable formatting for Submissions to this Evaluation.
        submission_receipt_message: Message to display to users upon successful submission to this Evaluation.

    Example: Create a new evaluation in a project
        &nbsp;
        Create a new evaluation on Synapse by storing an evaluation object with the required fields. If there are any fields missing, an error will be raised.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(
            name="My Challenge Evaluation",
            description="Evaluation for my data challenge",
            content_source="syn123456",
            submission_instructions_message="Submit CSV files only",
            submission_receipt_message="Thank you for your submission!",
        )
        created = evaluation.store()
        ```

    Example: Update an existing evaluation retrieved from Synapse by ID
        &nbsp;
        You can use the store method to create and update evaluations.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        evaluation.description = "Updated description for my evaluation"
        updated = evaluation.store()
        ```

    Example: Retrieve and update the ACL of an evaluation
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        acl = evaluation.get_acl()

        # Let's grant this team READ and SUBMIT permissions
        team_id = "123456"
        evaluation.update_acl(principal_id=team_id, access_type=["READ", "SUBMIT"])

        # Now let's revoke all permissions from this team
        team_to_revoke_from = "654321"
        evaluation.update_acl(principal_id=team_to_revoke_from, access_type=[])
        ```

    Example: Delete an evaluation
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        evaluation.delete()
        ```
    """

    id: Optional[str] = None
    """The unique immutable ID for this Evaluation."""

    etag: Optional[str] = None
    """Synapse employs an Optimistic Concurrency Control (OCC) scheme to handle concurrent updates.
    The eTag changes every time an Evaluation is updated; it is used to detect when a client's copy
    of an Evaluation is out-of-date."""

    name: Optional[str] = None
    """The name of this Evaluation."""

    description: Optional[str] = None
    """A text description of this Evaluation."""

    owner_id: Optional[str] = None
    """The ID of the Synapse user who created this Evaluation."""

    created_on: Optional[str] = None
    """The date on which Evaluation was created."""

    content_source: Optional[str] = None
    """The Synapse ID of the Entity to which this Evaluation belongs,
    e.g. a reference to a Synapse project."""

    submission_instructions_message: Optional[str] = None
    """Message to display to users detailing acceptable formatting for Submissions to this Evaluation."""

    submission_receipt_message: Optional[str] = None
    """Message to display to users upon successful submission to this Evaluation."""

    _last_persistent_instance: Optional["Evaluation"] = field(
        default=None, repr=False, compare=False
    )
    """The last persistent instance of this object. This is used to determine if the
    object has been changed and needs to be updated in Synapse."""

    def fill_from_dict(self, evaluation: dict) -> "Evaluation":
        """
        Converts the data coming from the Synapse Evaluation API into this datamodel.

        Arguments:
            evaluation: The data coming from the Synapse Evaluation API

        Returns:
            The Evaluation object instance.
        """
        self.id = evaluation.get("id", None)
        self.etag = evaluation.get("etag", None)
        self.name = evaluation.get("name", None)
        self.description = evaluation.get("description", None)
        self.owner_id = evaluation.get("ownerId", None)
        self.created_on = evaluation.get("createdOn", None)
        self.content_source = evaluation.get("contentSource", None)
        self.submission_instructions_message = evaluation.get(
            "submissionInstructionsMessage", None
        )
        self.submission_receipt_message = evaluation.get(
            "submissionReceiptMessage", None
        )

        return self

    @property
    def has_changed(self) -> bool:
        """Determines if the object has been newly created OR changed since last retrieval, and needs to be updated in Synapse."""
        return (
            not self._last_persistent_instance or self._last_persistent_instance != self
        )

    def _set_last_persistent_instance(self) -> None:
        """Stash the last time this object interacted with Synapse. This is used to
        determine if the object has been changed and needs to be updated in Synapse."""
        self._last_persistent_instance = replace(self)

    def _update_acl_permissions(
        self,
        principal_id: Union[str, int],
        access_type: List[str],
        acl: dict,
        synapse_client: Optional["Synapse"] = None,
    ) -> dict:
        """
        Updates the ACL permissions of this object for the given principal.

        Arguments:
            principal_id: The Synapse user or team ID to update permissions for
            access_type: List of permission strings to grant. If empty, the principal will be removed from the ACL.
            acl: The current ACL dictionary
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            The updated ACL dictionary
        """
        principal_id_int = int(principal_id)

        # If the access_type list is empty, remove the principal from the ACL
        if len(access_type) == 0:
            client = Synapse.get_client(synapse_client=synapse_client)
            client.logger.info(
                f"Principal ID {principal_id_int} will be removed from ACL due to empty access_type"
            )

            acl["resourceAccess"] = [
                permissions
                for permissions in acl["resourceAccess"]
                if int(permissions["principalId"]) != principal_id_int
            ]
            return acl

        # Update existing principal
        for permissions in acl["resourceAccess"]:
            if int(permissions["principalId"]) == principal_id_int:
                permissions["accessType"] = access_type
                return acl

        # Add new principal
        acl["resourceAccess"].append(
            {"principalId": principal_id_int, "accessType": access_type}
        )

        return acl

    def to_synapse_request(self, request_type: RequestType):
        """Creates a request body expected of the Synapse REST API for the Evaluation model.

        Arguments:
            request_type: The type of request to be made, either RequestType.CREATE or RequestType.UPDATE.

        Returns:
            A dictionary containing the request body for the specified request type.

        Raises:
            ValueError: If any required attributes are missing.
        """

        # These attributes are required in our PUT requests for creating or updating an evaluation
        required_attributes = [
            "name",
            "description",
            "content_source",
            "submission_instructions_message",
            "submission_receipt_message",
        ]

        # For "update" requests, add id and etag
        if request_type == RequestType.UPDATE:
            required_attributes.extend(["id", "etag"])

        for attribute in required_attributes:
            if not getattr(self, attribute):
                raise ValueError(
                    f"Your evaluation object is missing the '{attribute}' attribute. This attribute is required to {request_type.value} an evaluation"
                )

        # Build a request body for storing a brand new evaluation
        request_body = {
            "name": self.name,
            "description": self.description,
            "contentSource": self.content_source,
            "submissionInstructionsMessage": self.submission_instructions_message,
            "submissionReceiptMessage": self.submission_receipt_message,
        }

        # For UPDATE request types, add id and etag
        if request_type == RequestType.UPDATE:
            request_body["id"] = self.id
            request_body["etag"] = self.etag

        return request_body

    async def store_async(
        self, *, synapse_client: Optional["Synapse"] = None
    ) -> "Evaluation":
        """
        Create a new Evaluation or update an existing one in Synapse.

        If the Evaluation object has an ID and etag, it will be updated.
        Otherwise, a new Evaluation will be created.

        Arguments:
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            The created or updated Evaluation object.

        Raises:
            ValueError: If required fields are missing.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Creating a new evaluation
            &nbsp;
            Create a new evaluation on Synapse by storing an evaluation object with the required fields. If there are any fields missing, an error will be raised.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def create_evaluation():
                evaluation = await Evaluation(
                    name="My Challenge Evaluation",
                    description="Evaluation for my data challenge",
                    content_source="syn123456",
                    submission_instructions_message="Submit CSV files only",
                    submission_receipt_message="Thank you for your submission!"
                ).store_async()

                return evaluation

            created_evaluation = asyncio.run(create_evaluation())
            ```

        Example: Updating an existing evaluation
            &nbsp;
            You can use the store method to create and update evaluations.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def update_evaluation():
                evaluation = await Evaluation(id="9999999").get_async()
                evaluation.description = "Updated description for my evaluation"
                evaluation.submission_instructions_message = "New submission instructions"
                updated_evaluation = await evaluation.store_async()
                return updated_evaluation

            updated_evaluation = asyncio.run(update_evaluation())
            ```
        """

        from synapseclient.api.evaluation_services import create_or_update_evaluation

        # Get the client for logging
        client = Synapse.get_client(synapse_client=synapse_client)
        logger = client.logger

        # Set up OpenTelemetry tracing
        trace.get_current_span().set_attributes(
            {
                "synapse.name": self.name or "",
                "synapse.id": self.id or "",
            }
        )

        # CASE 1: No previous interaction with Synapse, so attempt to make a new evaluation
        if not self._last_persistent_instance:
            request_body = self.to_synapse_request(request_type=RequestType.CREATE)
            result = await create_or_update_evaluation(
                request_body=request_body,
                synapse_client=synapse_client,
            )

        # CASE 2: Previous interaction with Synapse, so attempt to update an existing evaluation
        elif self._last_persistent_instance:
            if self.has_changed:
                merge_dataclass_entities(
                    source=self._last_persistent_instance,
                    destination=self,
                    fields_to_preserve_from_source=[
                        "id",
                        "etag",
                        "content_source",
                        "owner_id",
                        "created_on",
                    ],
                    logger=logger,
                )
                request_body = self.to_synapse_request(request_type=RequestType.UPDATE)
                result = await create_or_update_evaluation(
                    request_body=request_body,
                    synapse_client=synapse_client,
                )
            else:
                logger.warning(
                    f"Evaluation {self.name} (ID: {self.id}) has not changed since last 'store' or 'get' event, so it will not be updated in Synapse. Please get the evaluation again if you want to refresh its state."
                )
                return self

        self.fill_from_dict(result)

        # Save the current state to track future changes
        self._set_last_persistent_instance()

        logger.debug(f"Saved Evaluation {self.name}, id: {self.id}")

        return self

    async def get_async(
        self, *, synapse_client: Optional["Synapse"] = None
    ) -> "Evaluation":
        """
        Get this Evaluation from Synapse by its ID or name.

        Arguments:
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            The retrieved Evaluation object.

        Raises:
            ValueError: If neither id nor name is set.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get an evaluation by ID and by name
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_evaluations():
                # Get an evaluation by ID
                evaluation_by_id = await Evaluation(id="9999999").get_async()

                # Get an evaluation by name
                evaluation_by_name = await Evaluation(name="My Challenge Evaluation").get_async()

                return evaluation_by_id, evaluation_by_name

            evaluation_by_id, evaluation_by_name = asyncio.run(get_evaluations())
            ```
        """
        from synapseclient.api.evaluation_services import get_evaluation

        if not self.id and not self.name:
            raise ValueError("Either id or name must be set to get an evaluation")

        retrieved_evaluation = await get_evaluation(
            evaluation_id=self.id,
            name=self.name,
            synapse_client=synapse_client,
        )

        self.fill_from_dict(retrieved_evaluation)

        # Save the current state to track future changes
        self._set_last_persistent_instance()

        return self

    async def delete_async(self, *, synapse_client: Optional["Synapse"] = None) -> None:
        """
        Delete this Evaluation from Synapse. ID must be set in order to delete the Evaluation.

        Arguments:
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Raises:
            ValueError: If evaluation_id is not set.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Delete an evaluation by ID
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def delete_evaluation():
                await Evaluation(id="9614112").delete_async()

            asyncio.run(delete_evaluation())
            ```

        Example: Get and then delete an evaluation
            &nbsp;
            If you do not have the ID of the evaluation, you can first retrieve it from Synapse by name. That will populate the ID attribute in your Evaluation object, at which point you can delete it.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_and_delete_evaluation():
                # First get the evaluation by name, so the ID attribute is set in your
                # Evaluation object, then delete it.
                evaluation = await Evaluation(name="My Challenge Evaluation").get_async()
                await evaluation.delete_async()

            asyncio.run(get_and_delete_evaluation())
            ```
        """
        from synapseclient.api.evaluation_services import delete_evaluation

        if not self.id:
            raise ValueError("id must be set to delete an evaluation")

        await delete_evaluation(
            evaluation_id=self.id,
            synapse_client=synapse_client,
        )

        # Clear the persistent instance since this object has been deleted
        self._last_persistent_instance = None

    async def get_acl_async(
        self, *, synapse_client: Optional["Synapse"] = None
    ) -> dict:
        """
        Get the access control list (ACL) governing this evaluation.

        Arguments:
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            The AccessControlList response object as a raw JSON dict.

        Raises:
            ValueError: If evaluation_id is not set.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get the ACL for an evaluation
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_evaluation_acl():
                # Get the evaluation first
                evaluation = await Evaluation(id="9999999").get_async()

                # Get the ACL for the evaluation
                acl = await evaluation.get_acl_async()
                return acl

            acl = asyncio.run(get_evaluation_acl())
            ```
        """
        from synapseclient.api.evaluation_services import get_evaluation_acl

        if not self.id:
            raise ValueError("id must be set to get evaluation ACL")

        return await get_evaluation_acl(
            evaluation_id=self.id,
            synapse_client=synapse_client,
        )

    async def update_acl_async(
        self,
        principal_id: Optional[Union[str, int]] = None,
        access_type: Optional[List[str]] = None,
        acl: Optional[dict] = None,
        *,
        synapse_client: Optional["Synapse"] = None,
    ) -> dict:
        """
        Update the access control list (ACL) for this evaluation.

        You can either: <br>
        1. Provide a `principal_id` and `access_type` list to update permissions for a specific user/team <br>
        2. Provide a complete ACL dictionary to update all permissions at once

        To remove a principal from the ACL completely, provide an empty list for access_type.

        The available access types are:

        - 'CREATE'
        - 'SUBMIT'
        - 'READ_PRIVATE_SUBMISSION'
        - 'DELETE_SUBMISSION'
        - 'UPDATE_SUBMISSION'
        - 'CHANGE_PERMISSIONS'
        - 'READ'
        - 'DELETE'
        - 'UPDATE'

        Arguments:
            principal_id: The Synapse user or team ID to update permissions for.
            access_type: List of permission strings to grant to the principal. If empty, the principal will be removed from the ACL.
            acl: A dictionary containing the complete ACL data to update. You can retrieve the current ACL using `get_acl_async()`.
                 If provided, principal_id and access_type are ignored.
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)`
                this will use the last created instance from the Synapse class constructor.

        Returns:
            The updated AccessControlList response object as a raw JSON dict.

        Raises:
            ValueError: If neither (principal_id and access_type) nor acl is provided, or if the ACL object is invalid.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Update permissions for a specific principal (user/team)
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def update_evaluation_permissions():
                # Get the evaluation first
                evaluation = await Evaluation(id="9999999").get_async()

                # Update permissions for user with ID 12345
                updated_acl = await evaluation.update_acl_async(
                    principal_id="12345",
                    access_type=["READ", "SUBMIT"]
                )
                return updated_acl

            updated_acl = asyncio.run(update_evaluation_permissions())
            ```

        Example: Remove a principal (user/team) from the ACL
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def remove_user_from_acl():
                # Get the evaluation first
                evaluation = await Evaluation(id="9999999").get_async()

                # Remove user with ID 12345 from the ACL by providing an empty list
                updated_acl = await evaluation.update_acl_async(
                    principal_id="12345",
                    access_type=[]
                )
                return updated_acl

            updated_acl = asyncio.run(remove_user_from_acl())
            ```

        Example: Update the entire ACL manually
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def update_evaluation_acl():
                # Get the evaluation first
                evaluation = await Evaluation(id="9999999").get_async()

                # Get the current ACL
                acl = await evaluation.get_acl_async()

                # Modify the ACL manually
                acl["resourceAccess"].append({
                    "principalId": 12345,
                    "accessType": ["READ", "SUBMIT"]
                })

                # Update with the modified ACL
                updated_acl = await evaluation.update_acl_async(acl=acl)
                return updated_acl

            updated_acl = asyncio.run(update_evaluation_acl())
            ```
        """
        from synapseclient.api.evaluation_services import update_evaluation_acl

        if not self.id:
            raise ValueError("id must be set to update evaluation ACL")

        # Case 1: Update permissions for specific principal
        if principal_id is not None and access_type is not None:
            access_type = [at.upper() for at in access_type]

            current_acl = await self.get_acl_async(synapse_client=synapse_client)

            updated_acl = self._update_acl_permissions(
                principal_id=principal_id,
                access_type=access_type,
                acl=current_acl,
                synapse_client=synapse_client,
            )

            return await update_evaluation_acl(
                acl=updated_acl,
                synapse_client=synapse_client,
            )

        # Case 2: Update entire ACL dictionary
        elif acl is not None:
            return await update_evaluation_acl(
                acl=acl,
                synapse_client=synapse_client,
            )

        else:
            raise ValueError(
                "Either (principal_id and access_type) or acl must be provided"
            )

    async def get_permissions_async(
        self,
        *,
        synapse_client: Optional["Synapse"] = None,
    ) -> dict:
        """
        Get the user permissions for this evaluation.

        Arguments:
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            dict: The permissions for the specified user.

        Raises:
            ValueError: If evaluation_id is not set.
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get permissions for the current user
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_evaluation_permissions():
                # Get the evaluation first
                evaluation = await Evaluation(id="9999999").get_async()

                # Get the permissions for the current user
                my_permissions = await evaluation.get_permissions_async()
                return my_permissions

            my_permissions = asyncio.run(get_evaluation_permissions())
            ```
        """
        from synapseclient.api.evaluation_services import get_evaluation_permissions

        if not self.id:
            raise ValueError("id must be set to get evaluation permissions")

        return await get_evaluation_permissions(
            evaluation_id=self.id,
            synapse_client=synapse_client,
        )

    @staticmethod
    async def get_all_evaluations_async(
        access_type: Optional[str] = None,
        active_only: Optional[bool] = None,
        evaluation_ids: Optional[List[str]] = None,
        offset: Optional[int] = None,
        limit: Optional[int] = None,
        *,
        synapse_client: Optional["Synapse"] = None,
    ) -> List["Evaluation"]:
        """
        Get a list of all Evaluations, within a given range.

        Arguments:
            access_type: The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ.
            active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
            evaluation_ids: An optional list of evaluation IDs to which the response is limited.
            offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
            limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            List[Evaluation]: A list of all evaluations.

        Raises:
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get all evaluations the user has at least READ access to
            &nbsp;
            A default call will return evaluations where the user has read access, without needing to specify access type.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_all_evaluations():
                all_evaluations = await Evaluation.get_all_evaluations_async()
                return all_evaluations

            all_evaluations = asyncio.run(get_all_evaluations())
            ```

        Example: Get only active evaluations with a limit
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_active_evaluations():
                active_evaluations = await Evaluation.get_all_evaluations_async(
                    active_only=True,
                    limit=20
                )
                return active_evaluations

            active_evaluations = asyncio.run(get_active_evaluations())
            ```

        Example: Get specific evaluations by ID
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_specific_evaluations():
                specific_evaluations = await Evaluation.get_all_evaluations_async(
                    evaluation_ids=["9999991", "9999992"]
                )
                return specific_evaluations

            specific_evaluations = asyncio.run(get_specific_evaluations())
            ```
        """
        from synapseclient.api.evaluation_services import get_all_evaluations

        result_dict = await get_all_evaluations(
            access_type=access_type,
            active_only=active_only,
            evaluation_ids=evaluation_ids,
            offset=offset,
            limit=limit,
            synapse_client=synapse_client,
        )
        items = (
            result_dict.get("results") if isinstance(result_dict, dict) else result_dict
        )
        evaluations: List[Evaluation] = []
        for item in items:
            evaluations.append(Evaluation().fill_from_dict(item))

        return evaluations

    @staticmethod
    async def get_available_evaluations_async(
        active_only: Optional[bool] = None,
        evaluation_ids: Optional[List[str]] = None,
        offset: Optional[int] = None,
        limit: Optional[int] = None,
        *,
        synapse_client: Optional["Synapse"] = None,
    ) -> List["Evaluation"]:
        """
        Get a list of Evaluations to which the user has SUBMIT permission, within a given range.

        Arguments:
            active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
            evaluation_ids: An optional list of evaluation IDs to which the response is limited.
            offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
            limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            List[Evaluation]: A list of available evaluations.

        Raises:
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get all evaluations where the current user has SUBMIT permission
            &nbsp;
            A default call will return evaluations where the user has SUBMIT permission, without needing to specify access type.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_available_evaluations():
                available_evaluations = await Evaluation.get_available_evaluations_async()
                return available_evaluations

            available_evaluations = asyncio.run(get_available_evaluations())
            ```

        Example: Get only active evaluations where the current user has SUBMIT permission
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_active_available_evaluations():
                active_available_evaluations = await Evaluation.get_available_evaluations_async(
                    active_only=True
                )
                return active_available_evaluations

            active_available_evaluations = asyncio.run(get_active_available_evaluations())
            ```

        Example: Get the first 5 evaluations where the current user has SUBMIT permission
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_limited_evaluations():
                limited_evaluations = await Evaluation.get_available_evaluations_async(
                    limit=5
                )
                return limited_evaluations

            limited_evaluations = asyncio.run(get_limited_evaluations())
            ```
        """
        from synapseclient.api.evaluation_services import get_available_evaluations

        result_dict = await get_available_evaluations(
            active_only=active_only,
            evaluation_ids=evaluation_ids,
            offset=offset,
            limit=limit,
            synapse_client=synapse_client,
        )
        items = (
            result_dict.get("results") if isinstance(result_dict, dict) else result_dict
        )
        evaluations: List[Evaluation] = []
        for item in items:
            evaluations.append(Evaluation().fill_from_dict(item))

        return evaluations

    @staticmethod
    async def get_evaluations_by_project_async(
        project_id: str,
        access_type: Optional[str] = None,
        active_only: Optional[bool] = None,
        evaluation_ids: Optional[List[str]] = None,
        offset: Optional[int] = None,
        limit: Optional[int] = None,
        *,
        synapse_client: Optional["Synapse"] = None,
    ) -> List["Evaluation"]:
        """
        Get Evaluations tied to a project.

        Arguments:
            project_id: The ID of the project (e.g., "syn123456").
            access_type: The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ.
            active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
            evaluation_ids: An optional list of evaluation IDs to which the response is limited.
            offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
            limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
            synapse_client: If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created
                            instance from the Synapse class constructor.

        Returns:
            List[Evaluation]: A list of Evaluations tied to the project.

        Raises:
            SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

        Example: Get all evaluations for a project
            &nbsp;
            The user must have at least READ access to the evaluations to retrieve the evaluations from a given project.
            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_project_evaluations():
                project_evaluations = await Evaluation.get_evaluations_by_project_async(
                    project_id="syn123456"
                )
                return project_evaluations

            project_evaluations = asyncio.run(get_project_evaluations())
            ```

        Example: Get only active evaluations for a project
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_active_project_evaluations():
                active_project_evaluations = await Evaluation.get_evaluations_by_project_async(
                    project_id="syn123456",
                    active_only=True
                )
                return active_project_evaluations

            active_project_evaluations = asyncio.run(get_active_project_evaluations())
            ```

        Example: Get a limited set of evaluations for a project
            &nbsp;

            ```python
            from synapseclient.models import Evaluation
            from synapseclient import Synapse
            import asyncio

            syn = Synapse()
            syn.login()

            async def get_limited_project_evaluations():
                limited_project_evaluations = await Evaluation.get_evaluations_by_project_async(
                    project_id="syn123456",
                    limit=5
                )
                return limited_project_evaluations

            limited_project_evaluations = asyncio.run(get_limited_project_evaluations())
            ```
        """
        from synapseclient.api.evaluation_services import get_evaluations_by_project

        result_dict = await get_evaluations_by_project(
            project_id=project_id,
            access_type=access_type,
            active_only=active_only,
            evaluation_ids=evaluation_ids,
            offset=offset,
            limit=limit,
            synapse_client=synapse_client,
        )
        items = (
            result_dict.get("results") if isinstance(result_dict, dict) else result_dict
        )
        evaluations: List[Evaluation] = []
        for item in items:
            evaluations.append(Evaluation().fill_from_dict(item))

        return evaluations

Functions¶

store ¶

store(*, synapse_client: Optional[Synapse] = None) -> Evaluation

Create a new Evaluation or update an existing one in Synapse.

If the Evaluation object has an ID and etag, it will be updated. Otherwise, a new Evaluation will be created.

PARAMETER	DESCRIPTION
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`Evaluation`	The created or updated Evaluation object.

RAISES	DESCRIPTION
`ValueError`	If required fields are missing.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Create a new evaluation in a project with ID "syn123456"

Create a new evaluation on Synapse by storing an evaluation object with the required fields. If there are any fields missing, an error will be raised.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(
    name="My Challenge Evaluation",
    description="Evaluation for my data challenge",
    content_source="syn123456",
    submission_instructions_message="Submit CSV files only",
    submission_receipt_message="Thank you for your submission!"
).store()

Update an evaluation that was retrieved from Synapse

You can use the store method to create and update evaluations.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
evaluation.description = "Updated description for my evaluation"
updated_evaluation = evaluation.store()

RAISES	DESCRIPTION
`ValueError`	If required fields are missing.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

def store(
    self,
    *,
    synapse_client: Optional[Synapse] = None,
) -> "Evaluation":
    """
    Create a new Evaluation or update an existing one in Synapse.

    If the Evaluation object has an ID and etag, it will be updated.
    Otherwise, a new Evaluation will be created.

    Arguments:
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        The created or updated Evaluation object.

    Raises:
        ValueError: If required fields are missing.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Create a new evaluation in a project with ID "syn123456"
        &nbsp;
        Create a new evaluation on Synapse by storing an evaluation object with the required fields. If there are any fields missing, an error will be raised.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(
            name="My Challenge Evaluation",
            description="Evaluation for my data challenge",
            content_source="syn123456",
            submission_instructions_message="Submit CSV files only",
            submission_receipt_message="Thank you for your submission!"
        ).store()
        ```

    Example: Update an evaluation that was retrieved from Synapse
        &nbsp;
        You can use the store method to create and update evaluations.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        evaluation.description = "Updated description for my evaluation"
        updated_evaluation = evaluation.store()
        ```

    Raises:
        ValueError: If required fields are missing.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return self

get ¶

get(*, synapse_client: Optional[Synapse] = None) -> Evaluation

Get this Evaluation from Synapse by its ID or name.

PARAMETER	DESCRIPTION
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`Evaluation`	The retrieved Evaluation object.

RAISES	DESCRIPTION
`ValueError`	If neither id nor name is set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get an evaluation by ID

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()

Get an evaluation by name

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(name="My Challenge Evaluation").get()

RAISES	DESCRIPTION
`ValueError`	If neither id nor name is set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

def get(
    self,
    *,
    synapse_client: Optional[Synapse] = None,
) -> "Evaluation":
    """
    Get this Evaluation from Synapse by its ID or name.

    Arguments:
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        The retrieved Evaluation object.

    Raises:
        ValueError: If neither id nor name is set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get an evaluation by ID
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        ```

    Example: Get an evaluation by name
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(name="My Challenge Evaluation").get()
        ```

    Raises:
        ValueError: If neither id nor name is set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return self

delete ¶

delete(*, synapse_client: Optional[Synapse] = None) -> None

Delete this Evaluation from Synapse. ID must be set in order to delete the Evaluation.

PARAMETER	DESCRIPTION
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`None`	None

RAISES	DESCRIPTION
`ValueError`	If evaluation_id is not set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Delete an evaluation by ID

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

Evaluation(id="9614112").delete()

Get and then delete an evaluation

If you do not have the ID of the evaluation, you can first retrieve it from Synapse by name. That will populate the ID attribute in your Evaluation object, at which point you can delete it.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

# First get the evaluation by name, so the ID attribute is set in your
# Evaluation object, then delete it.
evaluation = Evaluation(name="My Challenge Evaluation").get()
evaluation.delete()

RAISES	DESCRIPTION
`ValueError`	If evaluation_id is not set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

def delete(
    self,
    *,
    synapse_client: Optional[Synapse] = None,
) -> None:
    """
    Delete this Evaluation from Synapse. ID must be set in order to delete the Evaluation.

    Arguments:
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        None

    Raises:
        ValueError: If evaluation_id is not set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Delete an evaluation by ID
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        Evaluation(id="9614112").delete()
        ```

    Example: Get and then delete an evaluation
        &nbsp;
        If you do not have the ID of the evaluation, you can first retrieve it from Synapse by name.
        That will populate the ID attribute in your Evaluation object, at which point you can delete it.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        # First get the evaluation by name, so the ID attribute is set in your
        # Evaluation object, then delete it.
        evaluation = Evaluation(name="My Challenge Evaluation").get()
        evaluation.delete()
        ```

    Raises:
        ValueError: If evaluation_id is not set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return None

get_all_evaluations `staticmethod` ¶

get_all_evaluations(access_type: Optional[str] = None, active_only: Optional[bool] = None, evaluation_ids: Optional[List[str]] = None, offset: Optional[int] = None, limit: Optional[int] = None, *, synapse_client: Optional[Synapse] = None) -> List[Evaluation]

Get a list of all Evaluations, within a given range.

PARAMETER	DESCRIPTION
`access_type`	The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ. TYPE: `Optional[str]` DEFAULT: `None`
`active_only`	If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds. TYPE: `Optional[bool]` DEFAULT: `None`
`evaluation_ids`	An optional list of evaluation IDs to which the response is limited. TYPE: `Optional[List[str]]` DEFAULT: `None`
`offset`	The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0. TYPE: `Optional[int]` DEFAULT: `None`
`limit`	Limits the number of entities that will be fetched for this page. When null it will default to 10. TYPE: `Optional[int]` DEFAULT: `None`
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`List[Evaluation]`	List[Evaluation]: A list of all evaluations.

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get all evaluations the user has at least READ access to

A default call will return evaluations where the user has READ access, without needing to specify access type.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

all_evaluations = Evaluation.get_all_evaluations()

Get only active evaluations with a limit

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

active_evaluations = Evaluation.get_all_evaluations(
    active_only=True,
    limit=20
)

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

@staticmethod
def get_all_evaluations(
    access_type: Optional[str] = None,
    active_only: Optional[bool] = None,
    evaluation_ids: Optional[List[str]] = None,
    offset: Optional[int] = None,
    limit: Optional[int] = None,
    *,
    synapse_client: Optional[Synapse] = None,
) -> List["Evaluation"]:
    """
    Get a list of all Evaluations, within a given range.

    Arguments:
        access_type: The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ.
        active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
        evaluation_ids: An optional list of evaluation IDs to which the response is limited.
        offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
        limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        List[Evaluation]: A list of all evaluations.

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get all evaluations the user has at least READ access to
        &nbsp;
        A default call will return evaluations where the user has READ access, without needing to specify access type.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        all_evaluations = Evaluation.get_all_evaluations()
        ```

    Example: Get only active evaluations with a limit
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        active_evaluations = Evaluation.get_all_evaluations(
            active_only=True,
            limit=20
        )
        ```

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return []

get_available_evaluations `staticmethod` ¶

get_available_evaluations(active_only: Optional[bool] = None, evaluation_ids: Optional[List[str]] = None, offset: Optional[int] = None, limit: Optional[int] = None, *, synapse_client: Optional[Synapse] = None) -> List[Evaluation]

Get a list of Evaluations to which the user has SUBMIT permission, within a given range.

PARAMETER	DESCRIPTION
`active_only`	If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds. TYPE: `Optional[bool]` DEFAULT: `None`
`evaluation_ids`	An optional list of evaluation IDs to which the response is limited. TYPE: `Optional[List[str]]` DEFAULT: `None`
`offset`	The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0. TYPE: `Optional[int]` DEFAULT: `None`
`limit`	Limits the number of entities that will be fetched for this page. When null it will default to 10. TYPE: `Optional[int]` DEFAULT: `None`
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`List[Evaluation]`	List[Evaluation]: A list of available evaluations.

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get all evaluations where the current user has SUBMIT permission

A default call will return evaluations where the user has SUBMIT permission, without needing to specify access type.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

available_evaluations = Evaluation.get_available_evaluations()

Get the first 5 evaluations where the current user has SUBMIT permission

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

limited_evaluations = Evaluation.get_available_evaluations(
    limit=5
)

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

@staticmethod
def get_available_evaluations(
    active_only: Optional[bool] = None,
    evaluation_ids: Optional[List[str]] = None,
    offset: Optional[int] = None,
    limit: Optional[int] = None,
    *,
    synapse_client: Optional[Synapse] = None,
) -> List["Evaluation"]:
    """
    Get a list of Evaluations to which the user has SUBMIT permission, within a given range.

    Arguments:
        active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
        evaluation_ids: An optional list of evaluation IDs to which the response is limited.
        offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
        limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        List[Evaluation]: A list of available evaluations.

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get all evaluations where the current user has SUBMIT permission
        &nbsp;
        A default call will return evaluations where the user has SUBMIT permission, without needing to specify access type.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        available_evaluations = Evaluation.get_available_evaluations()
        ```

    Example: Get the first 5 evaluations where the current user has SUBMIT permission
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        limited_evaluations = Evaluation.get_available_evaluations(
            limit=5
        )
        ```

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return []

get_evaluations_by_project `staticmethod` ¶

get_evaluations_by_project(project_id: str, access_type: Optional[str] = None, active_only: Optional[bool] = None, evaluation_ids: Optional[List[str]] = None, offset: Optional[int] = None, limit: Optional[int] = None, *, synapse_client: Optional[Synapse] = None) -> List[Evaluation]

Get Evaluations tied to a project.

PARAMETER	DESCRIPTION
`project_id`	The ID of the project (e.g., "syn123456"). TYPE: `str`
`access_type`	The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ. TYPE: `Optional[str]` DEFAULT: `None`
`active_only`	If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds. TYPE: `Optional[bool]` DEFAULT: `None`
`evaluation_ids`	An optional list of evaluation IDs to which the response is limited. TYPE: `Optional[List[str]]` DEFAULT: `None`
`offset`	The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0. TYPE: `Optional[int]` DEFAULT: `None`
`limit`	Limits the number of entities that will be fetched for this page. When null it will default to 10. TYPE: `Optional[int]` DEFAULT: `None`
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`List[Evaluation]`	List[Evaluation]: A list of Evaluations tied to the project.

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get all evaluations for a project

The user must have at least READ access to the evaluations to retrieve the evaluations from a given project.

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

project_evaluations = Evaluation.get_evaluations_by_project(
    project_id="syn123456"
)

Get a limited set of evaluations for a project

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

limited_project_evaluations = Evaluation.get_evaluations_by_project(
    project_id="syn123456",
    limit=5
)

RAISES	DESCRIPTION
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

@staticmethod
def get_evaluations_by_project(
    project_id: str,
    access_type: Optional[str] = None,
    active_only: Optional[bool] = None,
    evaluation_ids: Optional[List[str]] = None,
    offset: Optional[int] = None,
    limit: Optional[int] = None,
    *,
    synapse_client: Optional[Synapse] = None,
) -> List["Evaluation"]:
    """
    Get Evaluations tied to a project.

    Arguments:
        project_id: The ID of the project (e.g., "syn123456").
        access_type: The type of access for the user to filter for, optional and defaults to ACCESS_TYPE.READ.
        active_only: If True then return only those evaluations with rounds defined and for which the current time is in one of the rounds.
        evaluation_ids: An optional list of evaluation IDs to which the response is limited.
        offset: The offset index determines where this page will start from. An index of 0 is the first entity. When null it will default to 0.
        limit: Limits the number of entities that will be fetched for this page. When null it will default to 10.
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        List[Evaluation]: A list of Evaluations tied to the project.

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get all evaluations for a project
        &nbsp;
        The user must have at least READ access to the evaluations to retrieve the evaluations from a given project.
        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        project_evaluations = Evaluation.get_evaluations_by_project(
            project_id="syn123456"
        )
        ```

    Example: Get a limited set of evaluations for a project
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        limited_project_evaluations = Evaluation.get_evaluations_by_project(
            project_id="syn123456",
            limit=5
        )
        ```

    Raises:
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return []

get_acl ¶

get_acl(*, synapse_client: Optional[Synapse] = None) -> Dict

Get the access control list (ACL) governing this evaluation.

PARAMETER	DESCRIPTION
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`Dict`	The ACL for this Evaluation.

RAISES	DESCRIPTION
`ValueError`	If evaluation_id is not set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get the ACL for an evaluation

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
acl = evaluation.get_acl()

Source code in synapseclient/models/protocols/evaluation_protocol.py

def get_acl(
    self,
    *,
    synapse_client: Optional[Synapse] = None,
) -> Dict:
    """
    Get the access control list (ACL) governing this evaluation.

    Arguments:
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        The ACL for this Evaluation.

    Raises:
        ValueError: If evaluation_id is not set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get the ACL for an evaluation
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        acl = evaluation.get_acl()
        ```
    """
    return {}

update_acl ¶

update_acl(principal_id: Optional[Union[str, int]] = None, access_type: Optional[List[str]] = None, acl: Optional[dict] = None, *, synapse_client: Optional[Synapse] = None) -> Dict

Update the access control list (ACL) for this evaluation.

You can either:
1. Provide a principal_id and access_type list to update permissions for a specific user/team
2. Provide a complete ACL dictionary to update all permissions at once

To remove a principal from the ACL completely, provide an empty list for access_type.

The available access types are:

'CREATE'
'SUBMIT'
'READ_PRIVATE_SUBMISSION'
'DELETE_SUBMISSION'
'UPDATE_SUBMISSION'
'CHANGE_PERMISSIONS'
'READ'
'DELETE'
'UPDATE'

PARAMETER	DESCRIPTION
`acl`	An AccessControlList object or dictionary containing the ACL data to update. TYPE: `Optional[dict]` DEFAULT: `None`
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`Dict`	The updated ACL.

RAISES	DESCRIPTION
`ValueError`	If neither (principal_id and access_type) nor acl is provided, or if the ACL object is invalid.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Update permissions for a specific principal (user/team)

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

# Get the evaluation first
evaluation = Evaluation(id="9999999").get()

# Update permissions for user with ID 12345
updated_acl = evaluation.update_acl(
    principal_id="12345",
    access_type=["READ", "SUBMIT"]
)

Remove a principal (user/team) from the ACL

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

# Get the evaluation first
evaluation = Evaluation(id="9999999").get()

# Remove user with ID 12345 from the ACL by providing an empty list
updated_acl = evaluation.update_acl(
    principal_id="12345",
    access_type=[]
)

Update the entire ACL manually

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

# Get the evaluation first
evaluation = Evaluation(id="9999999").get()

# Get the current ACL
acl = evaluation.get_acl()

# Modify the ACL manually
acl["resourceAccess"].append({
    "principalId": 12345,
    "accessType": ["READ", "SUBMIT"]
})

# Update with the modified ACL
updated_acl = evaluation.update_acl(acl=acl)

RAISES	DESCRIPTION
`ValueError`	If the ACL object is invalid or missing required fields.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

def update_acl(
    self,
    principal_id: Optional[Union[str, int]] = None,
    access_type: Optional[List[str]] = None,
    acl: Optional[dict] = None,
    *,
    synapse_client: Optional[Synapse] = None,
) -> Dict:
    """
    Update the access control list (ACL) for this evaluation.

    You can either: <br>
    1. Provide a `principal_id` and `access_type` list to update permissions for a specific user/team <br>
    2. Provide a complete ACL dictionary to update all permissions at once

    To remove a principal from the ACL completely, provide an empty list for access_type.

    The available access types are:

    - 'CREATE'
    - 'SUBMIT'
    - 'READ_PRIVATE_SUBMISSION'
    - 'DELETE_SUBMISSION'
    - 'UPDATE_SUBMISSION'
    - 'CHANGE_PERMISSIONS'
    - 'READ'
    - 'DELETE'
    - 'UPDATE'

    Arguments:
        acl: An AccessControlList object or dictionary containing the ACL data to update.
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        The updated ACL.

    Raises:
        ValueError: If neither (principal_id and access_type) nor acl is provided, or if the ACL object is invalid.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Update permissions for a specific principal (user/team)
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        # Get the evaluation first
        evaluation = Evaluation(id="9999999").get()

        # Update permissions for user with ID 12345
        updated_acl = evaluation.update_acl(
            principal_id="12345",
            access_type=["READ", "SUBMIT"]
        )
        ```

    Example: Remove a principal (user/team) from the ACL
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        # Get the evaluation first
        evaluation = Evaluation(id="9999999").get()

        # Remove user with ID 12345 from the ACL by providing an empty list
        updated_acl = evaluation.update_acl(
            principal_id="12345",
            access_type=[]
        )
        ```

    Example: Update the entire ACL manually
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        # Get the evaluation first
        evaluation = Evaluation(id="9999999").get()

        # Get the current ACL
        acl = evaluation.get_acl()

        # Modify the ACL manually
        acl["resourceAccess"].append({
            "principalId": 12345,
            "accessType": ["READ", "SUBMIT"]
        })

        # Update with the modified ACL
        updated_acl = evaluation.update_acl(acl=acl)
        ```

    Raises:
        ValueError: If the ACL object is invalid or missing required fields.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return {}

get_permissions ¶

get_permissions(*, synapse_client: Optional[Synapse] = None) -> Dict

Get the user permissions for this evaluation.

PARAMETER	DESCRIPTION
`synapse_client`	If not passed in and caching was not disabled by `Synapse.allow_client_caching(False)` this will use the last created instance from the Synapse class constructor. TYPE: `Optional[Synapse]` DEFAULT: `None`

RETURNS	DESCRIPTION
`Dict`	The permissions for the specified user.

RAISES	DESCRIPTION
`ValueError`	If evaluation_id is not set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Get permissions for the current user

from synapseclient.models import Evaluation
from synapseclient import Synapse

syn = Synapse()
syn.login()

evaluation = Evaluation(id="9999999").get()
my_permissions = evaluation.get_permissions()

RAISES	DESCRIPTION
`ValueError`	If evaluation_id is not set.
`SynapseHTTPError`	If the service rejects the request or an HTTP error occurs.

Source code in synapseclient/models/protocols/evaluation_protocol.py

def get_permissions(
    self,
    *,
    synapse_client: Optional[Synapse] = None,
) -> Dict:
    """
    Get the user permissions for this evaluation.

    Arguments:
        synapse_client: If not passed in and caching was not disabled by
            `Synapse.allow_client_caching(False)` this will use the last created
            instance from the Synapse class constructor.

    Returns:
        The permissions for the specified user.

    Raises:
        ValueError: If evaluation_id is not set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.

    Example: Get permissions for the current user
        &nbsp;

        ```python
        from synapseclient.models import Evaluation
        from synapseclient import Synapse

        syn = Synapse()
        syn.login()

        evaluation = Evaluation(id="9999999").get()
        my_permissions = evaluation.get_permissions()
        ```

    Raises:
        ValueError: If evaluation_id is not set.
        SynapseHTTPError: If the service rejects the request or an HTTP error occurs.
    """
    return {}

Evaluation¶

API reference¶

synapseclient.models.Evaluation dataclass ¶

Functions¶

store ¶

get ¶

delete ¶

get_all_evaluations staticmethod ¶

get_available_evaluations staticmethod ¶

get_evaluations_by_project staticmethod ¶

get_acl ¶

update_acl ¶

get_permissions ¶

synapseclient.models.Evaluation `dataclass` ¶

get_all_evaluations `staticmethod` ¶

get_available_evaluations `staticmethod` ¶

get_evaluations_by_project `staticmethod` ¶