asap.materialize package

Submodules

asap.materialize.materialize_sections module

Materialize Render sections using BetterBox client

exception asap.materialize.materialize_sections.MaterializeSectionsError

Bases: RenderModuleException

class asap.materialize.materialize_sections.MaterializeSectionsModule(input_data=None, schema_type=None, output_schema_type=None, args=None, logger_name='argschema.argschema_parser')

Bases: SparkModule

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type MaterializeSectionsParameters

default_output_schema

alias of MaterializeSectionsOutput

default_schema

alias of MaterializeSectionsParameters

classmethod get_args(**kwargs)

override to append to spark call

classmethod get_materialize_options(baseDataUrl=None, owner=None, project=None, stack=None, rootDirectory=None, width=None, height=None, maxLevel=None, fmt=None, maxOverviewWidthAndHeight=None, skipInterpolation=None, binaryMask=None, label=None, createIGrid=None, forceGeneration=None, renderGroup=None, numberOfRenderGroups=None, cleanUpPriorRun=None, filterListName=None, explainPlan=None, maxImageCacheGb=None, zValues=None, **kwargs)

run()

asap.materialize.render_downsample_sections module

class asap.materialize.render_downsample_sections.RenderSectionAtScale(schema_type=None, *args, **kwargs)

Bases: RenderModule

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type RenderSectionAtScaleParameters

default_output_schema

alias of RenderSectionAtScaleOutput

default_schema

alias of RenderSectionAtScaleParameters

classmethod downsample_specific_mipmapLevel(zvalues, input_stack=None, level=1, pool_size=1, image_directory=None, scale=None, imgformat=None, doFilter=None, fillWithNoise=None, filterListName=None, render=None, do_mp=True, bounds=None, **kwargs)

run()

class asap.materialize.render_downsample_sections.WithThreadPool(*args, **kwargs)

Bases: ThreadPool

asap.materialize.render_downsample_sections.check_stack_for_mipmaps(render, input_stack, zvalues)

asap.materialize.render_downsample_sections.create_tilespecs_without_mipmaps(render, montage_stack, level, z)

return tilespecs missing mipmaplevels above the specified level

asap.materialize.schemas module

class asap.materialize.schemas.Bounds(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: DefaultSchema

Bounds

key	description	default	field_type	json_type
minX	minX of bounds	None	Integer	int
maxX	maxX of bounds	None	Integer	int
minY	minY of bounds	None	Integer	int
maxY	maxY of bounds	None	Integer	int

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.DeleteMaterializedSectionsOutput(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: DefaultSchema

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.DeleteMaterializedSectionsParameters(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: ArgSchema

This schema is designed to be a schema_type for an ArgSchemaParser object

DeleteMaterializedSectionsParameters

key	description	default	field_type	json_type
input_json	file path of input json file	NA	InputFile	str
output_json	file path to output json file	NA	OutputFile	str
log_level	set the logging level of the module	ERROR	LogLevel	str
minZ	no description	(REQUIRED)	Integer	int
maxZ	no description	(REQUIRED)	Integer	int
basedir	base directory for materialization	(REQUIRED)	InputDir	str
pool_size	size of pool to use to delete files	NA	Integer	int
tilesource	no description	5	Integer	int

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.MaterializeSectionsOutput(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: DefaultSchema

MaterializeSectionsOutput

key	description	default	field_type	json_type
zValues	no description	(REQUIRED)	List	int
rootDirectory	no description	(REQUIRED)	InputDir	str
materializedDirectory	no description	(REQUIRED)	InputDir	str

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.MaterializeSectionsParameters(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: ArgSchema, MaterializedBoxParameters, ZRangeParameters, RenderParametersRenderWebServiceParameters, SparkParameters

This schema is designed to be a schema_type for an ArgSchemaParser object

MaterializeSectionsParameters

key	description	default	field_type	json_type
jarfile	spark jar to call java spark command	(REQUIRED)	String	str
className	spark class to call	(REQUIRED)	String	str
driverMemory	spark driver memory (important for local spark)	6g	String	str
memory	Memory required for spark job	NA	String	str
sparkhome	Spark home directory containing bin/spark_submit	(REQUIRED)	InputDir	str
spark_files	list of spark files to add to the spark submit command	NA	List	str
spark_conf	dictionary of key value pairs to add to spark_submit as –conf key=value	NA	Dict	?
masterUrl	spark master url. For local execution local[num_procs,num_retries]	(REQUIRED)	String	str
baseDataUrl	api endpoint url e.g. http://<host>[:port]/render-ws/v1	NA	String	str
owner	owner of target collection	NA	String	str
project	project fo target collection	NA	String	str
render	no description	NA	RenderClientParameters	dict
minZ	minimum Z integer	NA	Integer	int
maxZ	maximum Z integer	NA	Integer	int
stack	stack fromw which boxes will be materialized	(REQUIRED)	String	str
rootDirectory	directory in which materialization directory structure will be created (structure is <rootDirectory>/<project>/<stack>/<width>x<height>/<mipMapLevel>/<z>/<row>/<col>.<fmt>)	(REQUIRED)	OutputDir	str
width	width of flat rectangular tiles to generate	(REQUIRED)	Integer	int
height	height of flat rectangular tiles to generate	(REQUIRED)	Integer	int
maxLevel	maximum mipMapLevel to generate.	0	Integer	int
fmt	image format to generate mipmaps – PNG if not specified	NA	String	str
maxOverviewWidthAndHeight	maximum pixel size for width or height of overview image. If excluded or 0, no overview generated.	NA	Integer	int
skipInterpolation	whether to skip interpolation (e.g. DMG data)	NA	Boolean	bool
binaryMask	whether to use binary mask (e.g. DMG data)	NA	Boolean	bool
label	whether to generate single color tile labels rather than actual images	NA	Boolean	bool
createIGrid	whther to create an IGrid file	NA	Boolean	bool
forceGeneration	whether to regenerate existing tiles	NA	Boolean	bool
renderGroup	index (1-n) identifying coarse portion of layer to render	NA	Integer	int
numberOfRenderGroups	used in conjunction with renderGroup, total number of groups being used	NA	Integer	int
filterListName	Apply specified filter list to all renderings	NA	String	str
input_json	file path of input json file	NA	InputFile	str
output_json	file path to output json file	NA	OutputFile	str
log_level	set the logging level of the module	ERROR	LogLevel	str
cleanUpPriorRun	whether to regenerate most recently generated boxes of an identical plan. Useful for rerunning failed jobs.	NA	Boolean	bool
explainPlan	whether to perform a dry run, logging as partition stages are run but skipping materialization	NA	Boolean	bool
maxImageCacheGb	maximum image cache in GB of tilespec level 0 data to cache per core. Larger values may degrade performance due to JVM garbage collection.	2.0	Float	float
zValues	z indices to materialize	NA	List	int

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.RenderSectionAtScaleOutput(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: DefaultSchema

RenderSectionAtScaleOutput

key	description	default	field_type	json_type
image_directory	Directory in which the downsampled section images are saved	(REQUIRED)	InputDir	str
temp_stack	The temp stack that was used to generate the downsampled sections	(REQUIRED)	String	str

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.RenderSectionAtScaleParameters(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: RenderParameters

This schema is designed to be a schema_type for an ArgSchemaParser object

RenderSectionAtScaleParameters

key	description	default	field_type	json_type
input_json	file path of input json file	NA	InputFile	str
output_json	file path to output json file	NA	OutputFile	str
log_level	set the logging level of the module	ERROR	LogLevel	str
render	parameters to connect to render server	(REQUIRED)	RenderClientParameters	dict
input_stack	Input stack to make the downsample version of	(REQUIRED)	String	str
image_directory	Directory to save the downsampled sections	(REQUIRED)	OutputDir	str
imgformat	Image format (default - png)	png	String	str
doFilter	Apply filtering before rendering	True	Boolean	bool
fillWithNoise	Fill image with noise (default - False)	False	Boolean	bool
scale	scale of the downsampled sections	(REQUIRED)	Float	float
minZ	min Z to create the downsample section from	-1	Integer	int
maxZ	max Z to create the downsample section from	-1	Integer	int
filterListName	Apply specified filter list to all renderings	NA	String	str
bounds	no description	None	Bounds	dict
use_stack_bounds	Do you want to use stack bounds while downsampling?. Default=False	False	Boolean	bool
pool_size	number of parallel threads to use	20	Integer	int

opts = <marshmallow.schema.SchemaOpts object>

validate_data(data)

class asap.materialize.schemas.ValidateMaterializationOutput(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: DefaultSchema

ValidateMaterializationOutput

key	description	default	field_type	json_type
basedir	no description	(REQUIRED)	String	str
failures	no description	(REQUIRED)	List	str

opts = <marshmallow.schema.SchemaOpts object>

class asap.materialize.schemas.ValidateMaterializationParameters(extra=None, only=None, exclude=(), prefix='', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)

Bases: ArgSchema

This schema is designed to be a schema_type for an ArgSchemaParser object

ValidateMaterializationParameters

key	description	default	field_type	json_type
input_json	file path of input json file	NA	InputFile	str
output_json	file path to output json file	NA	OutputFile	str
log_level	set the logging level of the module	ERROR	LogLevel	str
minRow	minimum row to attempt to validate tiles. Will attempt to use stack bounds if None	NA	Integer	int
maxRow	no description	NA	Integer	int
minCol	no description	NA	Integer	int
maxCol	no description	NA	Integer	int
minZ	no description	(REQUIRED)	Integer	int
maxZ	no description	(REQUIRED)	Integer	int
ext	no description	png	String	str
basedir	base directory for materialization	(REQUIRED)	InputDir	str
pool_size	size of pool to use to investigate image validity	NA	Integer	int

opts = <marshmallow.schema.SchemaOpts object>

Module contents