spinn_front_end_common.utilities package¶

Subpackages¶

Submodules¶

spinn_front_end_common.utilities.base_database module¶

class spinn_front_end_common.utilities.base_database.BaseDatabase(database_file: str | None = None, *, read_only: bool = False, row_factory: type | None = <class 'sqlite3.Row'>, text_factory: type | None = <class 'memoryview'>)[source]¶

Bases: SQLiteDB

Specific implementation of the Database for SQLite 3.

There should only ever be a single Database Object in use at any time. In the case of application_graph_changed the first should closed and a new one created.

If 2 database objects where opened with the database_file they hold the same data. Unless someone else deletes that file.

Note

Not thread safe on the same database file! Threads can access different DBs just fine.

Parameters:: database_file (str) – The name of a file that contains (or will contain) an SQLite database holding the data. If omitted the default location will be used.

classmethod default_database_file() → str[source]¶

The path to the stand place the data.sqlite3 file will be stored

Return type:: str

spinn_front_end_common.utilities.constants module¶

spinn_front_end_common.utilities.constants.APPDATA_MAGIC_NUM = 2903706326¶: Application data magic number.

spinn_front_end_common.utilities.constants.APP_PTR_TABLE_BYTE_SIZE = 392¶: Size of data spec pointer table produced by DSE, in bytes.

spinn_front_end_common.utilities.constants.APP_PTR_TABLE_HEADER_BYTE_SIZE = 8¶: Size of header of data spec pointer table produced by DSE, in bytes. Note that the header consists of 2 uint32_t variables (magic_number, version)

spinn_front_end_common.utilities.constants.APP_PTR_TABLE_REGION_BYTE_SIZE = 12¶: Size of a region description in the pointer table. Note that the description consists of a pointer and 2 uint32_t variables: (pointer, checksum, n_words)

spinn_front_end_common.utilities.constants.BIT_FIELD_COMMS_SDRAM_TAG = 2¶: SDRAM Tags used for bitfield compressor

class spinn_front_end_common.utilities.constants.BufferingOperations(value)[source]¶

Bases: Enum

A listing of what SpiNNaker specific EIEIO commands there are.

BUFFER_READ = 0¶: Database handshake with external program

BUFFER_WRITE = 1¶: Host confirming data being read form SpiNNaker memory

spinn_front_end_common.utilities.constants.CLOCKS_PER_US = 200¶: The number of clock cycles per micro-second (at 200Mhz)

spinn_front_end_common.utilities.constants.COMPRESSOR_SDRAM_TAG = 1¶: SDRAM Tag used by the compressor to find the routing tables

spinn_front_end_common.utilities.constants.CORE_DATA_SDRAM_BASE_TAG = 200¶: Base SDRAM tag used by cores when loading data (tags 201-217 will be used by cores 1-17)

spinn_front_end_common.utilities.constants.DATA_SPECABLE_BASIC_SETUP_INFO_N_BYTES = 400¶: The number of words in the AbstractDataSpecable basic setup information. This is the amount required by the pointer table plus a SARK allocation.

spinn_front_end_common.utilities.constants.DEFAULT_BUFFER_SIZE_BEFORE_RECEIVE = 16384¶: The default size of a recording buffer before receive request is sent

spinn_front_end_common.utilities.constants.DSE_DATA_STRUCT_SIZE = 16¶: size of the on-chip DSE data structure required, in bytes

spinn_front_end_common.utilities.constants.DSE_VERSION = 65536¶: Version of the file produced by the DSE.

spinn_front_end_common.utilities.constants.MAX_DATABASE_PATH_LENGTH = 50000¶: Database file path maximum length for database notification messages. Note that this is not sent to SpiNNaker and so is not subject to the usual SDP limit.

spinn_front_end_common.utilities.constants.MAX_MEM_REGIONS = 32¶: Maximum number of memory regions in DSG virtual machine.

spinn_front_end_common.utilities.constants.MAX_POSSIBLE_BINARY_SIZE = 33792¶: the ITCM max limit for a binary

spinn_front_end_common.utilities.constants.MAX_SAFE_BINARY_SIZE = 32768¶: the ITCM max safe limit for a binary

spinn_front_end_common.utilities.constants.MAX_SIZE_OF_BUFFERED_REGION_ON_CHIP = 1048576¶: max size expected to be used by the reverse ip_tag multicast source during buffered operations.

spinn_front_end_common.utilities.constants.NOTIFY_PORT = 19999¶: The default local port that the toolchain listens on for the notification protocol.

spinn_front_end_common.utilities.constants.PARTITION_ID_FOR_MULTICAST_DATA_SPEED_UP = 'DATA_SPEED_UP_ROAD'¶: partition IDs preallocated to functionality

spinn_front_end_common.utilities.constants.SARK_PER_MALLOC_SDRAM_USAGE = 8¶: The number of bytes used by SARK per memory allocation

spinn_front_end_common.utilities.constants.SDRAM_BASE_ADDR = 1879048192¶: start of where SDRAM starts (either unbuffered or buffered)

spinn_front_end_common.utilities.constants.SDRAM_EDGE_BASE_TAG = 100¶: Base SDRAM tag used by SDRAM edges when allocating (allows up to 100 edges per chip)

spinn_front_end_common.utilities.constants.SIMULATION_N_BYTES = 12¶: The number of bytes used by the simulation interface. This is one word for the machine_time_step, one for the SDP port, and one for the application hash.

spinn_front_end_common.utilities.constants.SYSTEM_BYTES_REQUIREMENT = 412¶: The number of bytes used by the DSG and simulation interfaces

spinn_front_end_common.utilities.emergency_recovery module¶

spinn_front_end_common.utilities.emergency_recovery.emergency_recover_state_from_failure(vertex: AbstractHasAssociatedBinary, placement: Placement) → None[source]¶

Used to get at least some information out of a core when something goes badly wrong. Not a replacement for what abstract spinnaker base does.

Parameters:

vertex (AbstractHasAssociatedBinary) – The vertex to retrieve the IOBUF from if it is suspected as being dead
placement (Placement) – Where the vertex is located.

spinn_front_end_common.utilities.emergency_recovery.emergency_recover_states_from_failure() → None[source]¶: Used to get at least some information out of a core when something goes badly wrong. Not a replacement for what abstract spinnaker base does.

spinn_front_end_common.utilities.exceptions module¶

exception spinn_front_end_common.utilities.exceptions.BufferableRegionTooSmall[source]¶

Bases: SpinnFrontEndException

Raised when the SDRAM space of the region for buffered packets is too small to contain any packet at all.

exception spinn_front_end_common.utilities.exceptions.BufferedRegionNotPresent[source]¶

Bases: SpinnFrontEndException

Raised when trying to issue buffered packets for a region not managed.

exception spinn_front_end_common.utilities.exceptions.CantFindSDRAMToUseException[source]¶

Bases: SpinnFrontEndException

Raised when malloc and SDRAM stealing cannot occur.

exception spinn_front_end_common.utilities.exceptions.ConfigurationException[source]¶

Bases: SpinnFrontEndException

Raised when the front end determines a input parameter is invalid.

exception spinn_front_end_common.utilities.exceptions.DataSpecException[source]¶

Bases: SpinnFrontEndException

Raise when Data Specification did something wrong

exception spinn_front_end_common.utilities.exceptions.DatabaseException[source]¶

Bases: SpinnFrontEndException

Raise when something in a database failed.

exception spinn_front_end_common.utilities.exceptions.DsDatabaseException[source]¶

Bases: SpinnFrontEndException

Raise when a query in the Data Specification database failed.

exception spinn_front_end_common.utilities.exceptions.ExecutableFailedToStartException[source]¶

Bases: SpinnFrontEndException

Raised when an executable has not entered the expected state during start up.

exception spinn_front_end_common.utilities.exceptions.ExecutableFailedToStopException[source]¶

Bases: SpinnFrontEndException

Raised when an executable has not entered the expected state during execution.

exception spinn_front_end_common.utilities.exceptions.ExecutableNotFoundException[source]¶

Bases: SpinnFrontEndException

Raised when a specified executable could not be found.

exception spinn_front_end_common.utilities.exceptions.RallocException[source]¶

Bases: SpinnFrontEndException

Raised when there are not enough routing table entries.

exception spinn_front_end_common.utilities.exceptions.SpinnFrontEndException[source]¶

Bases: Exception

Raised when the front end detects an error.

spinn_front_end_common.utilities.helpful_functions module¶

spinn_front_end_common.utilities.helpful_functions.convert_string_into_chip_and_core_subset(cores: str | None) → CoreSubsets[source]¶

Translate a string list of cores into a core subset.

Parameters:: cores (str or None) – string representing down cores formatted as x,y,p[:x,y,p]*
Return type:: CoreSubsets

spinn_front_end_common.utilities.helpful_functions.convert_vertices_to_core_subset(vertices: Iterable[MachineVertex]) → CoreSubsets[source]¶

Converts vertices into core subsets.

Parameters:: vertices (iterable(MachineVertex)) – the vertices to convert to core subsets
Returns:: the CoreSubSets of the vertices
Return type:: CoreSubsets

spinn_front_end_common.utilities.helpful_functions.determine_flow_states(executable_types: Dict[ExecutableType, Any], no_sync_changes: int) → Tuple[Dict[ExecutableType, Collection[CPUState]], Dict[ExecutableType, Collection[CPUState]]][source]¶

Get the start and end states for these executable types.

Parameters:

executable_types (dict(ExecutableType,any)) – the execute types to locate start and end states from
no_sync_changes (int) – the number of times sync signals been sent

Returns:

dict of executable type to states.

Return type:

tuple(dict(ExecutableType,tuple(CPUState)), dict(ExecutableType,tuple(CPUState)))

spinn_front_end_common.utilities.helpful_functions.flood_fill_binary_to_spinnaker(binary: str) → int[source]¶

Flood fills a binary to SpiNNaker.

Parameters:: binary (str) – The name of the file containing the APLX binary to load
Returns:: the number of cores it was loaded onto
Return type:: int

spinn_front_end_common.utilities.helpful_functions.generate_unique_folder_name(folder: str, filename: str, extension: str) → str[source]¶

Generate a unique file name with a given extension in a given folder.

Parameters:

folder (str) – where to put this unique file
filename (str) – the name of the first part of the file without extension
extension (str) – extension of the file

Returns:

file path with a unique addition

Return type:

str

spinn_front_end_common.utilities.helpful_functions.get_defaultable_source_id(entry: MulticastRoutingEntry) → int[source]¶

Hack to support the source requirement for the router compressor on chip.

Parameters:: entry (MulticastRoutingEntry) – the multicast router table entry.
Returns:: return the source value
Return type:: int

spinn_front_end_common.utilities.helpful_functions.get_ethernet_chip(machine: Machine, board_address: str) → Chip[source]¶

Locate the chip with the given board IP address.

Parameters:

machine (Machine) – the SpiNNaker machine
board_address (str) – the board address to locate the chip of.

Returns:

The chip that supports that board address

Return type:

Chip

Raises:

ConfigurationException – when that board address has no chip associated with it

spinn_front_end_common.utilities.helpful_functions.get_region_base_address_offset(app_data_base_address: int, region: int) → int[source]¶

Find the address of the of a given region for the DSG.

Parameters:

app_data_base_address (int) – base address for the core
region (int) – the region ID we’re looking for

spinn_front_end_common.utilities.helpful_functions.locate_extra_monitor_mc_receiver(placement_x: int, placement_y: int) → DataSpeedUpPacketGatherMachineVertex[source]¶

Get the data speed up gatherer that can be used to talk to a particular chip. This will be on the same board.

Parameters:

placement_x (int) – The X coordinate of the reference chip
placement_y (int) – The Y coordinate of the reference chip

Return type:

DataSpeedUpPacketGatherMachineVertex

spinn_front_end_common.utilities.helpful_functions.locate_memory_region_for_placement(placement: Placement, region: int) → int[source]¶

Get the address of a region for a placement.

Parameters:

region (int) – the region to locate the base address of
placement (Placement) – the placement object to get the region address of

Returns:

the address

Return type:

int

spinn_front_end_common.utilities.helpful_functions.n_word_struct(n_words: int) → Struct[source]¶

Manages a precompiled cache of :py:class`~struct.Struct`s for parsing blocks of words. Thus, this:

data = n_word_struct(n_words).unpack(data_blob)

Is much like doing this:

data = struct.unpack(f"<{n_words}I", data_blob)

except quite a bit more efficient because things are shared including the cost of parsing the format.

Parameters:: n_words (int) – The number of SpiNNaker words to be handled.
Returns:: A struct for working with that many words.
Return type:: Struct

spinn_front_end_common.utilities.helpful_functions.read_data(x: int, y: int, address: int, length: int, data_format: str) → int[source]¶

Reads and converts a single data item from memory.

Parameters:

x (int) – chip x
y (int) – chip y
address (int) – base address of the SDRAM chip to read
length (int) – length to read
data_format (str) – the format to read memory (see struct.pack())

Returns:

whatever is produced by unpacking the data

Return type:

tuple

spinn_front_end_common.utilities.iobuf_extractor module¶

class spinn_front_end_common.utilities.iobuf_extractor.IOBufExtractor(executable_targets: ExecutableTargets | None = None, *, recovery_mode: bool = False, filename_template: str = 'iobuf_for_chip_{}_{}_processor_id_{}.txt', suppress_progress: bool = False)[source]¶

Bases: object

Extract the logging output buffers from the machine, and separates lines based on their prefix.

Parameters:

executable_targets (ExecutableTargets or None) – Which Binaries and core to extract from. None to extract from all.
recovery_mode (bool)
filename_template (str)
suppress_progress (bool)

extract_iobuf() → Tuple[Sequence[str], Sequence[str]][source]¶

Perform the extraction of IOBUF.

Returns:: error_entries, warn_entries
Return type:: tuple(list(str),list(str))

spinn_front_end_common.utilities.math_constants module¶

random math constants

spinn_front_end_common.utilities.sqlite_db module¶

class spinn_front_end_common.utilities.sqlite_db.SQLiteDB(database_file: str | None = None, *, read_only: bool = False, ddl_file: str | None = None, row_factory: ~typing.Type[~sqlite3.Row] | ~typing.Type[tuple] | None = <class 'sqlite3.Row'>, text_factory: type | None = <class 'memoryview'>, case_insensitive_like: bool = True, timeout: float = 5.0)[source]¶

Bases: object

General support class for SQLite databases. This handles a lot of the low-level detail of setting up a connection.

Basic usage (with the default row type):

with SQLiteDB("db_file.sqlite3") as database:
    with database.transaction() as cursor:
        for row in cursor.execute("SELECT thing FROM ..."):
            print(row["thing"])

This class is designed to be either used as above or by subclassing. See the SQLite SQL documentation for details of how to write queries, and the Python sqlite3 module for how to do parameter binding.

Note

If you plan to use the WAL journal mode for the DB, you are recommended to set this up in the DDL file via:

PRAGMA journal_mode=WAL;

This is because the journal mode is persistent for database. For details, see the SQLite documentation on the write-ahead log.

Parameters:

database_file (str) – The name of a file that contains (or will contain) an SQLite database holding the data. If omitted, an unshared in-memory database will be used (suitable only for testing).
read_only (bool) – Whether the database is in read-only mode. When the database is in read-only mode, it must already exist.
ddl_file (str or None) – The name of a file (typically containing SQL DDL commands used to create the tables) to be evaluated against the database before this object completes construction. If None, nothing will be evaluated. You probably don’t want to specify a DDL file at the same time as setting read_only=True.
row_factory (Callable or None) – Callable used to create the rows of result sets. Either tuple or sqlite3.Row (default); can be None to use the DB driver default.
text_factory (Callable or None) – Callable used to create the Python values of non-numeric columns in result sets. Usually memoryview (default) but should be str when you’re expecting string results; can be None to use the DB driver default.
case_insensitive_like (bool) – Whether we want the LIKE matching operator to be case-sensitive or case-insensitive (default).
timeout (float) – How many seconds the connection should wait before raising an OperationalError when a table is locked. If another connection opens a transaction to modify a table, that table will be locked until the transaction is committed. Default five seconds.
synchronisation (Synchronisation) – The synchronisation level. Doesn’t normally need to be altered.

close() → None[source]¶: Finalises and closes the database.

cursor() → Cursor[source]¶: Gets the cursor created by the with statement

fetchone() → Row[source]¶

Gets the fetchone from the last query run

Raises:: DatabaseException – If there is no cursor. Typically because database was used outside of a with

property lastrowid: int¶

Gets the lastrowid from the last query run/ execute

Return type:: int
Raises:: DatabaseException – If there is no cursor. Typically because database was used outside of a with

property rowcount: int¶

Gets the rowcount from the last query run/ execute

Return type:: int
Raises:: DatabaseException – If there is no cursor. Typically because database was used outside of a with

spinn_front_end_common.utilities.system_control_logic module¶

spinn_front_end_common.utilities.system_control_logic.run_system_application(executable_cores: ExecutableTargets, app_id: int, read_algorithm_iobuf: bool, check_for_success_function: Callable[[ExecutableTargets], bool] | None, cpu_end_states: FrozenSet[CPUState], needs_sync_barrier: bool, filename_template: str, binaries_to_track: List[str] | None = None, progress_bar: ProgressBar | None = None, logger: FormatAdapter | None = None, timeout: float | None = None) → None[source]¶

Executes the given _system_ application. Used for on-chip expander, compressors, etc.

Parameters:

executable_cores (ExecutableTargets) – the cores to run the executable on.
app_id (int) – the app-id for the executable
read_algorithm_iobuf (bool) – whether to report IOBUFs
check_for_success_function (callable) – function used to check success; expects executable_cores, transceiver as inputs
cpu_end_states (frozenset(CPUState)) – the states that a successful run is expected to terminate in
needs_sync_barrier (bool) – whether a sync barrier is needed
filename_template (str) – the IOBUF filename template.
binaries_to_track (list(str)) – A list of binary names to check for exit state. Or None for all binaries
progress_bar (ProgressBar or None) – Possible progress bar to update. end() will be called after state checked
logger (Logger) – If provided and IOBUF is extracted, will be used to log errors and warnings
timeout (float or None) – Number of seconds to wait before force stopping, or None to wait forever

Raises:

SpinnmanException – If one should arise from the underlying SpiNNMan calls

spinn_front_end_common.utilities.utility_calls module¶

Utility calls for interpreting bits of the DSG

spinn_front_end_common.utilities.utility_calls.check_file_exists(path: str) → None[source]¶

Check to see a file that should exist does

Raises an exception if it does not

Parameters:: path – path to file that should exist
Raises:: FileNotFoundError – If the path does not exists

spinn_front_end_common.utilities.utility_calls.get_region_base_address_offset(app_data_base_address: int, region: int) → int[source]¶

Find the address of the of a given region for the DSG.

Parameters:

app_data_base_address (int) – base address for the core
region (int) – the region ID we’re looking for

spinn_front_end_common.utilities.utility_calls.get_report_writer(processor_chip_x: int, processor_chip_y: int, processor_id: int, use_run_number: bool = False) → TextIO | None[source]¶

Check if text reports are needed, and if so initialise the report writer to send down to DSG.

Parameters:

processor_chip_x (int) – X coordinate of the chip
processor_chip_y (int) – Y coordinate of the chip
processor_id (int) – The processor ID
use_run_number (bool) – If set the directory will include the run number

Returns:

the report_writer_object, or None if not reporting

Return type:

FileIO or None

spinn_front_end_common.utilities.utility_calls.open_scp_connection(chip_x: int, chip_y: int, chip_ip_address: str) → SCAMPConnection[source]¶

Create an SCP connection to the given Ethernet-enabled chip. SpiNNaker will not be configured to map that connection to a tag; that is the caller’s responsibility.

Parameters:

chip_x (int) – X coordinate of the Ethernet-enabled chip to connect to.
chip_y (int) – Y coordinate of the Ethernet-enabled chip to connect to.
chip_ip_address (str) – IP address of the Ethernet-enabled chip to connect to.

Return type:

SCAMPConnection

spinn_front_end_common.utilities.utility_calls.parse_old_spalloc(spalloc_server: str, spalloc_port: int, spalloc_user: str) → Tuple[str, int, str][source]¶

Parse a URL to the old-style service. This may take the form:

spalloc://user@spalloc.host.example.com:22244

The leading spalloc:// is the mandatory part (as is the actual host name). If the port and user are omitted, the defaults given in the other arguments are used (or default defaults).

A bare hostname can be used instead. If that’s the case (i.e., there’s no spalloc:// prefix) then the port and user are definitely used.

Parameters:

spalloc_server (str) – Hostname or URL
spalloc_port (int) – Default port
spalloc_user (str) – Default user

Returns:

hostname, port, username

Return type:

tuple(str,int,str)

spinn_front_end_common.utilities.utility_calls.pick_core_for_system_placement(system_placements: Placements, chip: Chip) → int[source]¶

Get a core number for use putting a system placement on a chip.

Parameters:

system_placements (Placements) – Already-made system placements
chip (Chip) – Chip of interest

Returns:

a core that a system placement could be put on

Return type:

int

spinn_front_end_common.utilities.utility_calls.retarget_tag(connection: SpallocEIEIOListener | SpallocEIEIOConnection | SCAMPConnection, x: int, y: int, tag: int, ip_address: str | None = None, strip: bool = True) → None[source]¶

Make a tag deliver to the given connection.

Parameters:

connection (UDPConnection) – The connection to deliver to.
x (int) – The X coordinate of the Ethernet-enabled chip we are sending the message to.
y (int) – The Y coordinate of the Ethernet-enabled chip we are sending the message to.
tag (int) – The ID of the tag to retarget.
ip_address (str) – What IP address to send the message to. If None, the connection is assumed to be connected to a specific board already.
strip (bool) – Whether the tag should strip the SDP header before sending to the connection.

spinn_front_end_common.utilities package¶

Subpackages¶

Submodules¶

spinn_front_end_common.utilities.base_database module¶

spinn_front_end_common.utilities.constants module¶

spinn_front_end_common.utilities.emergency_recovery module¶

spinn_front_end_common.utilities.exceptions module¶

spinn_front_end_common.utilities.helpful_functions module¶

spinn_front_end_common.utilities.iobuf_extractor module¶

spinn_front_end_common.utilities.math_constants module¶

spinn_front_end_common.utilities.sqlite_db module¶

spinn_front_end_common.utilities.system_control_logic module¶

spinn_front_end_common.utilities.utility_calls module¶

Module contents¶

Table of Contents

Previous topic

Next topic

This Page