spinn_front_end_common.utilities package¶
Subpackages¶
- spinn_front_end_common.utilities.connections package
- spinn_front_end_common.utilities.database package
- spinn_front_end_common.utilities.notification_protocol package
- spinn_front_end_common.utilities.report_functions package
- spinn_front_end_common.utilities.scp package
- spinn_front_end_common.utilities.utility_objs package
Submodules¶
spinn_front_end_common.utilities.base_database module¶
-
class
spinn_front_end_common.utilities.base_database.
BaseDatabase
(database_file=None, *, read_only=False, row_factory=<class 'sqlite3.Row'>, text_factory=<class 'memoryview'>)[source]¶ Bases:
spinn_front_end_common.utilities.sqlite_db.SQLiteDB
,spinn_utilities.abstract_context_manager.AbstractContextManager
Specific implementation of the Database for SQLite 3.
There should only ever be a single Database Object in use at any time. In the case of application_graph_changed the first should closed and a new one created.
If 2 database objects where opened with the database_file they hold the same data. Unless someone else deletes that file.
Note
Not thread safe on the same database file! Threads can access different DBs just fine.
Parameters: database_file (str) – The name of a file that contains (or will contain) an SQLite database holding the data. If omitted the default location will be used.
spinn_front_end_common.utilities.constants module¶
-
spinn_front_end_common.utilities.constants.
BIT_FIELD_COMMS_SDRAM_TAG
= 2¶ SDRAM Tags used for bitfield compressor
-
class
spinn_front_end_common.utilities.constants.
BUFFERING_OPERATIONS
[source]¶ Bases:
enum.Enum
A listing of what SpiNNaker specific EIEIO commands there are.
-
BUFFER_READ
= 0¶ Database handshake with external program
-
BUFFER_WRITE
= 1¶ Host confirming data being read form SpiNNaker memory
-
-
spinn_front_end_common.utilities.constants.
CLOCKS_PER_US
= 200¶ The number of clock cycles per micro-second (at 200Mhz)
-
spinn_front_end_common.utilities.constants.
COMPRESSOR_SDRAM_TAG
= 1¶ SDRAM Tag used by the compressor to find the routing tables
-
spinn_front_end_common.utilities.constants.
CORE_DATA_SDRAM_BASE_TAG
= 200¶ Base SDRAM tag used by cores when loading data (tags 201-217 will be used by cores 1-17)
-
spinn_front_end_common.utilities.constants.
DATA_SPECABLE_BASIC_SETUP_INFO_N_BYTES
= 400¶ The number of words in the AbstractDataSpecable basic setup information. This is the amount required by the pointer table plus a SARK allocation.
-
spinn_front_end_common.utilities.constants.
DEFAULT_BUFFER_SIZE_BEFORE_RECEIVE
= 16384¶ The default size of a recording buffer before receive request is sent
-
spinn_front_end_common.utilities.constants.
DSE_DATA_STRUCT_SIZE
= 16¶ size of the on-chip DSE data structure required, in bytes
-
spinn_front_end_common.utilities.constants.
MAX_DATABASE_PATH_LENGTH
= 50000¶ Database file path maximum length for database notification messages. Note that this is not sent to SpiNNaker and so is not subject to the usual SDP limit.
-
spinn_front_end_common.utilities.constants.
MAX_POSSIBLE_BINARY_SIZE
= 33792¶ the ITCM max limit for a binary
-
spinn_front_end_common.utilities.constants.
MAX_SAFE_BINARY_SIZE
= 32768¶ the ITCM max safe limit for a binary
-
spinn_front_end_common.utilities.constants.
MAX_SIZE_OF_BUFFERED_REGION_ON_CHIP
= 1048576¶ max size expected to be used by the reverse ip_tag multicast source during buffered operations.
-
spinn_front_end_common.utilities.constants.
NOTIFY_PORT
= 19999¶ The default local port that the toolchain listens on for the notification protocol.
-
spinn_front_end_common.utilities.constants.
PARTITION_ID_FOR_MULTICAST_DATA_SPEED_UP
= 'DATA_SPEED_UP_ROAD'¶ partition IDs preallocated to functionality
-
spinn_front_end_common.utilities.constants.
SARK_PER_MALLOC_SDRAM_USAGE
= 8¶ The number of bytes used by SARK per memory allocation
-
class
spinn_front_end_common.utilities.constants.
SDP_PORTS
[source]¶ Bases:
enum.Enum
SDP port handling output buffering data streaming.
-
EXTRA_MONITOR_CORE_DATA_IN_SPEED_UP
= 6¶ Extra monitor core inbound data transfer protocol See
SpeedupInSCPCommands
-
EXTRA_MONITOR_CORE_DATA_SPEED_UP
= 5¶ Extra monitor core outbound data transfer protocol
-
EXTRA_MONITOR_CORE_REINJECTION
= 4¶ Extra monitor core reinjection control protocol. See
ReinjectorSCPCommands
-
INPUT_BUFFERING_SDP_PORT
= 1¶ Command port for the buffered in functionality.
-
OUTPUT_BUFFERING_SDP_PORT
= 2¶ Command port for the buffered out functionality.
-
RUNNING_COMMAND_SDP_PORT
= 3¶ Command port for resetting runtime, etc. See
SDP_RUNNING_MESSAGE_CODES
-
-
class
spinn_front_end_common.utilities.constants.
SDP_RUNNING_MESSAGE_CODES
[source]¶ Bases:
enum.Enum
Codes for sending control messages to spin1_api.
-
SDP_CLEAR_IOBUF_CODE
= 9¶
-
SDP_NEW_RUNTIME_ID_CODE
= 7¶
-
SDP_STOP_ID_CODE
= 6¶
-
SDP_UPDATE_PROVENCE_REGION_AND_EXIT
= 8¶
-
-
spinn_front_end_common.utilities.constants.
SDRAM_BASE_ADDR
= 1879048192¶ start of where SDRAM starts (either unbuffered or buffered)
-
spinn_front_end_common.utilities.constants.
SDRAM_EDGE_BASE_TAG
= 100¶ Base SDRAM tag used by SDRAM edges when allocating (allows up to 100 edges per chip)
-
spinn_front_end_common.utilities.constants.
SIMULATION_N_BYTES
= 12¶ The number of bytes used by the simulation interface. This is one word for the machine_time_step, one for the SDP port, and one for the application hash.
-
spinn_front_end_common.utilities.constants.
SYSTEM_BYTES_REQUIREMENT
= 412¶ The number of bytes used by the DSG and simulation interfaces
spinn_front_end_common.utilities.emergency_recovery module¶
-
spinn_front_end_common.utilities.emergency_recovery.
emergency_recover_state_from_failure
(vertex, placement)[source]¶ Used to get at least some information out of a core when something goes badly wrong. Not a replacement for what abstract spinnaker base does.
Parameters: - txrx (Transceiver) – The transceiver.
- vertex (AbstractHasAssociatedBinary) – The vertex to retrieve the IOBUF from if it is suspected as being dead
- placement (Placement) – Where the vertex is located.
-
spinn_front_end_common.utilities.emergency_recovery.
emergency_recover_states_from_failure
()[source]¶ Used to get at least some information out of a core when something goes badly wrong. Not a replacement for what abstract spinnaker base does.
Parameters: executable_targets (ExecutableTargets) – The what/where mapping
spinn_front_end_common.utilities.exceptions module¶
-
exception
spinn_front_end_common.utilities.exceptions.
BufferableRegionTooSmall
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when the SDRAM space of the region for buffered packets is too small to contain any packet at all.
-
exception
spinn_front_end_common.utilities.exceptions.
BufferedRegionNotPresent
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when trying to issue buffered packets for a region not managed.
-
exception
spinn_front_end_common.utilities.exceptions.
CantFindSDRAMToUseException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when malloc and SDRAM stealing cannot occur.
-
exception
spinn_front_end_common.utilities.exceptions.
ConfigurationException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when the front end determines a input parameter is invalid.
-
exception
spinn_front_end_common.utilities.exceptions.
DsDatabaseException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raise when a query in the Data Specification database failed.
-
exception
spinn_front_end_common.utilities.exceptions.
ExecutableFailedToStartException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when an executable has not entered the expected state during start up.
-
exception
spinn_front_end_common.utilities.exceptions.
ExecutableFailedToStopException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when an executable has not entered the expected state during execution.
-
exception
spinn_front_end_common.utilities.exceptions.
ExecutableNotFoundException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when a specified executable could not be found.
-
exception
spinn_front_end_common.utilities.exceptions.
RallocException
[source]¶ Bases:
spinn_front_end_common.utilities.exceptions.SpinnFrontEndException
Raised when there are not enough routing table entries.
spinn_front_end_common.utilities.helpful_functions module¶
-
spinn_front_end_common.utilities.helpful_functions.
convert_string_into_chip_and_core_subset
(cores)[source]¶ Translate a string list of cores into a core subset.
Parameters: cores (str or None) – string representing down cores formatted as x,y,p[:x,y,p]* Return type: CoreSubsets
-
spinn_front_end_common.utilities.helpful_functions.
convert_vertices_to_core_subset
(vertices)[source]¶ Converts vertices into core subsets.
Parameters: vertices (iterable(MachineVertex)) – the vertices to convert to core subsets Returns: the CoreSubSets of the vertices Return type: CoreSubsets
-
spinn_front_end_common.utilities.helpful_functions.
determine_flow_states
(executable_types, no_sync_changes)[source]¶ Get the start and end states for these executable types.
Parameters: - executable_types (dict(ExecutableType,any)) – the execute types to locate start and end states from
- no_sync_changes (int) – the number of times sync signals been sent
Returns: dict of executable type to states.
Return type: tuple(dict(ExecutableType,CPUState), dict(ExecutableType,CPUState))
-
spinn_front_end_common.utilities.helpful_functions.
flood_fill_binary_to_spinnaker
(binary)[source]¶ Flood fills a binary to SpiNNaker.
Parameters: binary (str) – The name of the file containing the APLX binary to load Returns: the number of cores it was loaded onto Return type: int
-
spinn_front_end_common.utilities.helpful_functions.
generate_unique_folder_name
(folder, filename, extension)[source]¶ Generate a unique file name with a given extension in a given folder.
Parameters: Returns: file path with a unique addition
Return type:
-
spinn_front_end_common.utilities.helpful_functions.
get_defaultable_source_id
(entry)[source]¶ Hack to support the source requirement for the router compressor on chip.
Parameters: entry (MulticastRoutingEntry) – the multicast router table entry. Returns: return the source value Return type: int
-
spinn_front_end_common.utilities.helpful_functions.
get_ethernet_chip
(machine, board_address)[source]¶ Locate the chip with the given board IP address.
Parameters: Returns: The chip that supports that board address
Return type: Raises: ConfigurationException – when that board address has no chip associated with it
-
spinn_front_end_common.utilities.helpful_functions.
get_region_base_address_offset
(app_data_base_address, region)[source]¶ Find the address of the of a given region for the DSG.
Parameters:
-
spinn_front_end_common.utilities.helpful_functions.
locate_extra_monitor_mc_receiver
(placement_x, placement_y)[source]¶ Get the data speed up gatherer that can be used to talk to a particular chip. This will be on the same board.
Parameters: Return type:
-
spinn_front_end_common.utilities.helpful_functions.
locate_memory_region_for_placement
(placement, region)[source]¶ Get the address of a region for a placement.
Parameters: - region (int) – the region to locate the base address of
- placement (Placement) – the placement object to get the region address of
- transceiver (Transceiver) – the python interface to the SpiNNaker machine
Returns: the address
Return type:
-
spinn_front_end_common.utilities.helpful_functions.
n_word_struct
(n_words)[source]¶ Manages a precompiled cache of :py:class`~struct.Struct`s for parsing blocks of words. Thus, this:
data = n_word_struct(n_words).unpack(data_blob)
Is much like doing this:
data = struct.unpack(f"<{n_words}I", data_blob)
except quite a bit more efficient because things are shared including the cost of parsing the format.
Parameters: n_words (int) – The number of SpiNNaker words to be handled. Returns: A struct for working with that many words. Return type: Struct
-
spinn_front_end_common.utilities.helpful_functions.
read_data
(x, y, address, length, data_format)[source]¶ Reads and converts a single data item from memory.
Parameters: Returns: whatever is produced by unpacking the data
Return type:
-
spinn_front_end_common.utilities.helpful_functions.
write_address_to_user0
(x, y, p, address)[source]¶ Writes the given address into the user_0 register of the given core.
Parameters:
spinn_front_end_common.utilities.iobuf_extractor module¶
-
class
spinn_front_end_common.utilities.iobuf_extractor.
IOBufExtractor
(executable_targets=None, recovery_mode=False, filename_template='iobuf_for_chip_{}_{}_processor_id_{}.txt', suppress_progress=False)[source]¶ Bases:
object
Extract the logging output buffers from the machine, and separates lines based on their prefix.
Parameters:
spinn_front_end_common.utilities.math_constants module¶
random math constants
spinn_front_end_common.utilities.sqlite_db module¶
-
class
spinn_front_end_common.utilities.sqlite_db.
Isolation
[source]¶ Bases:
enum.Enum
Transaction isolation levels for
SQLiteDB.transaction()
.-
DEFERRED
= 'DEFERRED'¶ Standard transaction type; postpones holding a lock until required.
-
EXCLUSIVE
= 'EXCLUSIVE'¶ Take a write lock immediately. This is the strongest lock type.
-
IMMEDIATE
= 'IMMEDIATE'¶ Take the lock immediately; this may be a read-lock that gets upgraded.
-
-
class
spinn_front_end_common.utilities.sqlite_db.
SQLiteDB
(database_file=None, *, read_only=False, ddl_file=None, row_factory=<class 'sqlite3.Row'>, text_factory=<class 'memoryview'>, case_insensitive_like=True)[source]¶ Bases:
spinn_utilities.abstract_context_manager.AbstractContextManager
General support class for SQLite databases. This handles a lot of the low-level detail of setting up a connection.
Basic usage (with the default row type):
with SQLiteDB("db_file.sqlite3") as db: with db.transaction() as cursor: for row in cursor.execute("SELECT thing FROM ..."): print(row["thing"])
This class is designed to be either used as above or by subclassing. See the SQLite SQL documentation for details of how to write queries, and the Python
sqlite3
module for how to do parameter binding.Parameters: - database_file (str) – The name of a file that contains (or will contain) an SQLite database holding the data. If omitted, an unshared in-memory database will be used (suitable only for testing).
- read_only (bool) – Whether the database is in read-only mode. When the database is in read-only mode, it must already exist.
- ddl_file (str or None) – The name of a file (typically containing SQL DDL commands used to
create the tables) to be evaluated against the database before this
object completes construction. If
None
, nothing will be evaluated. You probably don’t want to specify a DDL file at the same time as settingread_only=True
. - row_factory (Callable or None) – Callable used to create the rows of result sets.
Either
tuple
orsqlite3.Row
(default); can beNone
to use the DB driver default. - text_factory (Callable or None) – Callable used to create the Python values of non-numeric columns
in result sets. Usually
memoryview
(default) but should bestr
when you’re expecting string results; can beNone
to use the DB driver default. - case_insensitive_like (bool) – Whether we want the
LIKE
matching operator to be case-sensitive or case-insensitive (default).
-
connection
¶ The underlying SQLite database connection.
Warning
If you’re using this a lot, consider contacting the SpiNNaker Software Team with details of your use case so we can extend the relevant core class to support you. Normally it is better to use
transaction()
to obtain a cursor with appropriate transactional guards.Return type: Connection Raises: AttributeError – if the database connection has been closed
-
pragma
(pragma_name, value)[source]¶ Set a database
PRAGMA
. See the SQLite PRAGMA documentation for details.Parameters:
-
transaction
(isolation_level=None)[source]¶ Get a context manager that manages a transaction on the database. The value of the context manager is a
Cursor
. This means you can do this:with db.transaction() as cursor: cursor.execute(...)
Parameters: isolation_level (Isolation) – The transaction isolation level.
Note
This sets it for the connection! Can usually be not specified.
Return type: ContextManager(Cursor)
spinn_front_end_common.utilities.system_control_logic module¶
-
spinn_front_end_common.utilities.system_control_logic.
run_system_application
(executable_cores, app_id, read_algorithm_iobuf, check_for_success_function, cpu_end_states, needs_sync_barrier, filename_template, binaries_to_track=None, progress_bar=None, logger=None, timeout=None)[source]¶ Executes the given _system_ application. Used for on-chip expanders, compressors, etc.
Parameters: - executable_cores (ExecutableTargets) – the cores to run the executable on.
- app_id (int) – the app-id for the executable
- read_algorithm_iobuf (bool) – whether to report IOBUFs
- check_for_success_function (callable) – function used to check success; expects executable_cores, transceiver as inputs
- cpu_end_states (set(CPUState)) – the states that a successful run is expected to terminate in
- needs_sync_barrier (bool) – whether a sync barrier is needed
- filename_template (str) – the IOBUF filename template.
- binaries_to_track (list(str)) – A list of binary names to check for exit state. Or None for all binaries
- progress_bar (ProgressBar or None) – Possible progress bar to update. end() will be called after state checked
- logger (Logger) – If provided and IOBUF is extracted, will be used to log errors and warnings
- timeout (float or None) – Number of seconds to wait before force stopping, or None to wait forever
Raises: SpinnmanException – If one should arise from the underlying SpiNNMan calls
spinn_front_end_common.utilities.utility_calls module¶
Utility calls for interpreting bits of the DSG
-
spinn_front_end_common.utilities.utility_calls.
get_data_spec_and_file_writer_filename
(processor_chip_x, processor_chip_y, processor_id, application_run_time_report_folder='TEMP')[source]¶ Encapsulates the creation of the DSG writer and the file paths.
Parameters: Returns: the filename of the data writer and the data specification object
Return type:
-
spinn_front_end_common.utilities.utility_calls.
get_region_base_address_offset
(app_data_base_address, region)[source]¶ Find the address of the of a given region for the DSG.
Parameters:
-
spinn_front_end_common.utilities.utility_calls.
get_report_writer
(processor_chip_x, processor_chip_y, processor_id)[source]¶ Check if text reports are needed, and if so initialise the report writer to send down to DSG.
Parameters: Returns: the report_writer_object, or None if not reporting
Return type:
-
spinn_front_end_common.utilities.utility_calls.
open_scp_connection
(chip_x, chip_y, chip_ip_address)[source]¶ Create an SCP connection to the given Ethernet-enabled chip. SpiNNaker will not be configured to map that connection to a tag; that is the caller’s responsibility.
Parameters: Return type:
-
spinn_front_end_common.utilities.utility_calls.
parse_old_spalloc
(spalloc_server, spalloc_port=22244, spalloc_user='unknown user')[source]¶ Parse a URL to the old-style service. This may take the form:
spalloc://user@spalloc.host.example.com:22244The leading
spalloc://
is the mandatory part (as is the actual host name). If the port and user are omitted, the defaults given in the other arguments are used (or default defaults).A bare hostname can be used instead. If that’s the case (i.e., there’s no
spalloc://
prefix) then the port and user are definitely used.Parameters: Returns: hostname, port, username
Return type:
-
spinn_front_end_common.utilities.utility_calls.
retarget_tag
(connection, x, y, tag, ip_address=None, strip=True)[source]¶ Make a tag deliver to the given connection.
Parameters: - connection (UDPConnection) – The connection to deliver to.
- x (int) – The X coordinate of the Ethernet-enabled chip we are sending the message to.
- y (int) – The Y coordinate of the Ethernet-enabled chip we are sending the message to.
- tag (int) – The ID of the tag to retarget.
- ip_address (str) – What IP address to send the message to. If
None
, the connection is assumed to be connected to a specific board already. - strip (bool) – Whether the tag should strip the SDP header before sending to the connection.