timApp.util package

Subpackages

• timApp.util.flask package
  • Submodules
  • timApp.util.flask.cache module
  • timApp.util.flask.filters module
  • timApp.util.flask.requesthelper module
  • timApp.util.flask.responsehelper module
  • timApp.util.flask.search module
  • timApp.util.flask.typedblueprint module
  • timApp.util.flask.user_agent module
  • Module contents

Submodules

timApp.util.answerutil module

Answer-related routes.

class timApp.util.answerutil.AnswerPeriodOptions(period: timApp.util.answerutil.PeriodOptions = <PeriodOptions.WHENEVER: 'whenever'>, period_from: datetime.datetime | None = None, period_to: datetime.datetime | None = None)

Bases: object

period: timApp.util.answerutil.PeriodOptions = 'whenever'

period_from: datetime.datetime | None = None

period_to: datetime.datetime | None = None

class timApp.util.answerutil.PeriodOptions(value)

Bases: enum.Enum

An enumeration.

DAY = 'day'

MONTH = 'month'

OTHER = 'other'

SINCE_LAST = 'sincelast'

WEEK = 'week'

WHENEVER = 'whenever'

timApp.util.answerutil.get_answer_period(task_ids: list[timApp.plugin.taskid.TaskId], doc_ids: set[int], options: timApp.util.answerutil.AnswerPeriodOptions) → tuple[datetime.datetime, datetime.datetime]

Returns start and end of a period for answer results.

Parameters

• task_ids – Task ids containing the answers.

• doc_ids – Documents containing the answers.

• options –

Returns

Return “from”-period and “to”-period.

timApp.util.answerutil.task_ids_to_strlist(ids: list[timApp.plugin.taskid.TaskId]) → list[str]

timApp.util.filemodehelper module

timApp.util.filemodehelper.change_permission_and_retry(func: Callable[[str], None], path: str, exc_info: Any) → None

timApp.util.get_fields module

class timApp.util.get_fields.GetFieldsAccess(value)

Bases: enum.Enum

An enumeration.

AllowAlwaysNonTeacher = 2

AllowMaybeNonTeacher = 1

RequireTeacher = 0

static from_bool(b: bool)

class timApp.util.get_fields.MembershipFilter(value)

Bases: enum.Enum

An enumeration.

All = 'all'

Current = 'current'

Deleted = 'deleted'

class timApp.util.get_fields.RequestedGroups(groups: list[timApp.user.usergroup.UserGroup], include_all_answered: bool = False)

Bases: object

static from_name_list(group_names: list[str])

groups: list[timApp.user.usergroup.UserGroup]

include_all_answered: bool = False

class timApp.util.get_fields.TallyField(field: str, doc_id: int | None, datetime_start: datetime.datetime | None, datetime_end: datetime.datetime | None, default_doc: timApp.document.docinfo.DocInfo)

Bases: object

In Jsrunner, represents the “tally:” type of field.

datetime_end: datetime.datetime | None

datetime_start: datetime.datetime | None

default_doc: timApp.document.docinfo.DocInfo

property doc_and_field

doc_id: int | None

property effective_doc_id

field: str

property grouping_key

static try_parse(s: str, default_doc: timApp.document.docinfo.DocInfo) → Optional[timApp.util.get_fields.TallyField]

class timApp.util.get_fields.UserFieldObj

Bases: TypedDict

fields: dict[str, Union[str, float, NoneType]]

styles: Any

user: timApp.user.user.User

timApp.util.get_fields.chunks(l: list, n: int)

timApp.util.get_fields.get_fields_and_users(u_fields: list[str], requested_groups: timApp.util.get_fields.RequestedGroups, d: timApp.document.docinfo.DocInfo, current_user: timApp.user.user.User, view_ctx: timApp.document.viewcontext.ViewContext, autoalias: bool = False, add_missing_fields: bool = False, access_option: timApp.util.get_fields.GetFieldsAccess = GetFieldsAccess.RequireTeacher, member_filter_type: timApp.util.get_fields.MembershipFilter = MembershipFilter.Current, user_filter=None) → tuple[list[timApp.util.get_fields.UserFieldObj], dict[str, str], list[str], list[timApp.user.usergroup.UserGroup] | None]

Return fielddata, aliases, field_names

Parameters

• view_ctx – The view context.

• user_filter – Additional filter to use.

• member_filter_type – Whether to use all, current or deleted users in groups.

• u_fields – list of fields to be used

• requested_groups – requested user groups to be used

• d – default document

• current_user – current users, check his rights to fields

• autoalias – if true, give automatically from d1 same as would be from d1 = d1

• add_missing_fields – return estimated field even if it wasn’t given previously

• access_option – option specifying who is allowed to access fields (non-teachers, teachers)

Returns

fielddata, aliases, field_names

timApp.util.get_fields.get_tally_field_values(d: timApp.document.docinfo.DocInfo, doc_map: dict[int, timApp.document.docinfo.DocInfo], group_filter, join_relation, tally_fields: list[tuple[timApp.util.get_fields.TallyField, str | None]], view_ctx: timApp.document.viewcontext.ViewContext, user_ctx: timApp.document.usercontext.UserContext)

timApp.util.git_utils module

timApp.util.git_utils.get_current_branch() → str

timApp.util.git_utils.get_latest_commit_timestamp() → str

timApp.util.git_utils.try_fix_safe_dir_config() → None

timApp.util.logger module

Sets up logging for the application.

timApp.util.logger.enable_loggers() → None

timApp.util.logger.log_debug(message: str) → None

timApp.util.logger.log_error(message: str) → None

timApp.util.logger.log_info(message: str) → None

timApp.util.logger.log_warning(message: str) → None

timApp.util.logger.setup_logging(app: flask.app.Flask) → None

timApp.util.pdftools module

Stamping and merging pdf files with qpdf and pdflatex. Visa Naukkarinen

class timApp.util.pdftools.Attachment(url: str, macro: str = 'unknown', error: str = '', selected: bool = True)

Bases: object

to_json()

exception timApp.util.pdftools.AttachmentNotAPdfError(file_path: pathlib.Path = PosixPath('.'))

Bases: timApp.util.pdftools.PdfError

Raised when at least one file in input is not a pdf.

exception timApp.util.pdftools.AttachmentNotFoundError(file_path: str = '')

Bases: timApp.util.pdftools.PdfError

Raised when at least one pdf file in input data is missing.

class timApp.util.pdftools.AttachmentStampData(file_path: pathlib.Path = PosixPath('.'), date: str = '', attachment: str = '', issue: str | int = '', text: str = '')

Bases: object

Contains data to create stamp for one attachment.

validate()

Checks stamp data parameters and attachment file validity. :return: None.

exception timApp.util.pdftools.CompressionError

Bases: Exception

exception timApp.util.pdftools.MergeListEmptyError

Bases: timApp.util.pdftools.PdfError

Raised if empty list given to merge.

exception timApp.util.pdftools.ModelStampInvalidError(file_path: pathlib.Path = PosixPath('.'))

Bases: timApp.util.pdftools.PdfError

Raised if model tex file for creating stamps is broken.

exception timApp.util.pdftools.ModelStampMissingError(file_path: pathlib.Path = PosixPath('.'))

Bases: timApp.util.pdftools.PdfError

Raised if model tex file for creating stamps can’t be found.

exception timApp.util.pdftools.PdfError

Bases: Exception

Inherited by all the other custom errors pdftools-module uses.

exception timApp.util.pdftools.StampDataEmptyError

Bases: timApp.util.pdftools.PdfError

Raised if input data is an empty list.

exception timApp.util.pdftools.StampDataInvalidError(reason='', item='')

Bases: timApp.util.pdftools.PdfError

Raised if stamp data type is wrong.

exception timApp.util.pdftools.StampDataMissingAttributeError(attribute: str = '', item: str = '')

Bases: timApp.util.pdftools.PdfError

Raised when stamp data is missing one or more required attributes.

exception timApp.util.pdftools.StampFileNotFoundError(file_path: pathlib.Path = PosixPath('.'))

Bases: timApp.util.pdftools.PdfError

Raised when stamp to use is missing.

exception timApp.util.pdftools.StampFormatInvalidError(stampformat: str = '')

Bases: timApp.util.pdftools.PdfError

Raised if stampformat (from TIM-document settings) is incorrectly formatted.

exception timApp.util.pdftools.SubprocessError(cmd: str = '')

Bases: timApp.util.pdftools.PdfError

Raised when subprocesses (qpdf, pdflatex, possibly others) return error code or otherwise raise exception.

exception timApp.util.pdftools.TempFolderNotFoundError(folder_path: pathlib.Path = PosixPath('.'))

Bases: timApp.util.pdftools.PdfError

Raised if the folder for temporary files is missing.

timApp.util.pdftools.check_pdf_validity(pdf_path: pathlib.Path) → None

Raises error if pdf file doesn’t exist or isn’t really a pdf.

Parameters

pdf_path – Pdf to check.

Returns

None if not interrupted by error.

timApp.util.pdftools.check_stamp_data_validity(stamp_data_list: list[timApp.util.pdftools.AttachmentStampData]) → None

Raises a specific error if stamp_data is invalid.

Parameters

stamp_data_list – List of objects containing the stamp data.

Returns

None, but will raise error if something invalid.

timApp.util.pdftools.compress_pdf(f: timApp.upload.uploadedfile.UploadedFile)

timApp.util.pdftools.compress_pdf_if_not_already(f: timApp.upload.uploadedfile.UploadedFile)

timApp.util.pdftools.create_stamp(model_path: pathlib.Path, work_dir: pathlib.Path, stamp_name: str, text: str, remove_pdflatex_files: bool = False) → pathlib.Path

Creates a stamp pdf file with given text into temp folder.

Parameters

• model_path – Model stamp tex file’s complete path; contains ‘%TEXT_HERE’ to locate the stamp text area.

• work_dir – The folder where stamp output and temp files will be.

• stamp_name – Name of the stamp and temp files (no file extension needed).

• text – LaTeX-escaped text displayed in the stamp.

• remove_pdflatex_files – If true, newly created .aux, .log, .out and .tex files will be deleted.

Returns

Complete path of the created stamp pdf file.

timApp.util.pdftools.create_tex_file(content: str, folder: pathlib.Path = PosixPath('/tmp')) → pathlib.Path

Creates tex file with random name and input content.

Parameters

• content – LaTeX content in string.format.

• folder – Folder where new file will be added.

Returns

Path-object for the new file.

timApp.util.pdftools.escape_tex(text: str)

Escapes special characters in a TeX string. Idea taken from https://stackoverflow.com/a/25875504.

Parameters

text – A plain text message.

Returns

The message escaped to appear correctly in LaTeX.

timApp.util.pdftools.get_attachments_from_pars(paragraphs: list[timApp.document.docparagraph.DocParagraph]) → list[timApp.util.pdftools.Attachment]

Goes through paragraphs and gets attachments from showPdf-macros. Checks file validity with qpdf.

Parameters

paragraphs – Document paragraphs.

Returns

List of pdf paths and whether the list is complete.

timApp.util.pdftools.get_stamp_text(item: timApp.util.pdftools.AttachmentStampData, text_format: str) → str

Gives formatted stamp text; note: may not work properly with non-ascii.

Parameters

• item – AttachmentStampData with ‘date’,’attachment’ and ‘issue’ attributes or alternatively just ‘text’.

• text_format – Formatting for filename, meeting date, attachment letter and issue/list number.

Returns

Either contents of ‘text’ attribute or a formatted string.

timApp.util.pdftools.is_pdf_producer_ghostscript(f: timApp.upload.uploadedfile.UploadedFile)

timApp.util.pdftools.merge_pdfs(pdf_file_list: list[timApp.upload.uploadedfile.UploadedFile], output_path: pathlib.Path)

Merges a list of PDFs using qpdf.

Parameters

• pdf_file_list – List of the uploaded files (as objects) to merge.

• output_path – Merged output file path.

timApp.util.pdftools.parse_error(message: str) → str

Shortens known long PDFtk error messages and logs unexpected ones.

Parameters

message – Error message string from qpdf.

Returns

Pre-written error message.

timApp.util.pdftools.parse_tim_url(par_file: str) → pathlib.Path | None

Parses TIM-links to point to the corresponding file on the server. Note: Changes in upload folder need to be updated here as well.

Parameters

par_file – Link from the macro.

Returns

TIM upload file path if the link was within TIM, otherwise pass on unchanged.

timApp.util.pdftools.qpdf_popen(args: list[str], timeout_seconds=30) → None

Calls Popen with args list for QPDF, checks QPDF-specific error code and raises error if timeouted.

Parameters

• args – List of arguments.

• timeout_seconds – Timeout after which error is raised.

Returns

None.

timApp.util.pdftools.remove_temp_files(dir_path: pathlib.Path, temp_file_name: str, ext_list: list[str]) → None

Deletes temp files created for the stamping process.

Parameters

• dir_path – Temp-file folder path.

• temp_file_name – Common part of the names.

• ext_list – List of extensions (after the common part) for files to remove.

Returns

None.

timApp.util.pdftools.run_ghostscript(inputfile: pathlib.Path, outputfile: pathlib.Path) → str

timApp.util.pdftools.stamp_pdf(pdf_path: pathlib.Path, stamp_path: pathlib.Path, output_path: pathlib.Path, remove_stamp: bool = False) → pathlib.Path

Creates a new stamped pdf file (with stamp overlay on each page).

Parameters

• pdf_path – Path of the pdf to stamp.

• stamp_path – Path of the stamp file.

• output_path – Path of the new stamped pdf.

• remove_stamp – Delete stamp file after use.

Returns

output_path

timApp.util.pdftools.stamp_pdfs(stamp_data: list[timApp.util.pdftools.AttachmentStampData], stamp_model_path: pathlib.Path = PosixPath('static/tex/stamp_model.tex'), stamp_text_format: str = 'Kokous {date}\n\nLIITE {attachment} lista {issue}') → list[pathlib.Path]

Creates new stamps and stamps the corresponding pdfs based on the data in a list of AttachmentStampData objects.

Parameters

• stamp_data – List of objects containing pdf-paths and stamp-attributes.

• stamp_model_path – Tex file to be used as model for stamps.

• stamp_text_format – Formatting for stamp text, with attributes: file, date, attachment and issue.

Returns

List of stamped pdf paths.

timApp.util.pdftools.test_pdf(pdf_path: pathlib.Path, timeout_seconds: int = 300) → str

Test pdf file suitability for qpdf.

Parameters

• pdf_path – Pdf to test.

• timeout_seconds – Timeout after which error is raised.

Returns

Return error message (empty string if no error).

timApp.util.rndutils module

Functions to produce random lists. For documentation, see: https://tim.jyu.fi/view/tim/ohjeita/satunnaistus

class timApp.util.rndutils.SeedClass(seed: int, extraseed: int = 0)

Bases: object

extraseed: int = 0

seed: int

timApp.util.rndutils.fix_jso(jso: str) → str

If jso does not start with [ and two to make it list of lists.

timApp.util.rndutils.get_int_list(myrandom: random.Random, jso: str) → list[int]

Returns list of random ints from given interval.

Parameters

• myrandom – random number generator

• jso – string to find the values

Returns

list of random ints ints

timApp.util.rndutils.get_rands_as_dict(attrs: dict, rnd_seed: Optional[Union[str, int, timApp.util.rndutils.SeedClass]], state: Optional[tuple[int, ...]] = None) → tuple[dict | None, Union[str, int, timApp.util.rndutils.SeedClass, NoneType], tuple[int, ...] | None]

Returns a dict of random numbers variables (each is a list of random numbers).

Parameters

• attrs – dict where may be attrinute rndnames:”rnd1,rnd2,..,rndn”. Of no names, “rnd” is assumed

• rnd_seed – seed to initialize the generator

:param state of last used generator :return: dict of random variables

timApp.util.rndutils.get_rands_as_str(attrs: dict, rnd_seed: Optional[Union[str, int, timApp.util.rndutils.SeedClass]], state: Optional[tuple[int, ...]] = None) → tuple[str, Union[str, int, timApp.util.rndutils.SeedClass, NoneType], tuple[int, ...] | None]

Returns a Jinja2 str of random numbers variables (each is a list of random numbers).

Parameters

• attrs – dict where may be attrinute rndnames:”rnd1,rnd2,..,rndn”. Of no names, “rnd” is assumed

• rnd_seed – seed to initialize the generator

:param state of last used generator :return: Jinja 2 str of random variables

timApp.util.rndutils.get_rnds(attrs: dict, name: str = 'rnd', rnd_seed: Optional[Union[str, int, timApp.util.rndutils.SeedClass]] = None, state: Optional[tuple[int, ...]] = None) → tuple[list[float] | list[int] | None, Union[str, int, timApp.util.rndutils.SeedClass, NoneType], tuple[int, ...] | None]

Returns list of random numbers based on attribute name (def: rnd) and rnd_seed.

Parameters

• attrs – dict of attributes

• name – name in attribute dict to use as instructions for the random numbers

• rnd_seed – random number initializion seed, if seed is None, use time

:param state of last used generator :return: list of random numbers and used seed

timApp.util.rndutils.get_sample_list(myrandom: random.Random, jso: str) → list[int]

Returns a list of unique ints from the given interval.

Parameters

• myrandom – random number generator

• jso – string to find the values

Returns

list of unique ints

timApp.util.rndutils.get_uniform_list(myrandom: random.Random, jso: str) → list[float]

Returns list of uniformely distributed random floats from given interval.

Parameters

• myrandom – random number generator

• jso – string to find the values

Returns

list of random ints ints

timApp.util.rndutils.myhash(s: str) → int

Simple hash function to give always same hash for same input.

Parameters

s – string to hash

Returns

simple hash

timApp.util.rndutils.repeat_rnd(list_func: Callable[[random.Random, str], list[T]], myrandom: random.Random, jso: str) → list[T] | None

Parameters

• list_func – function to produce random list

• myrandom – random number generator

• jso – string to parse instructions

Returns

list of random numbers

timApp.util.rndutils.sep_n_and_jso(jso: str) → tuple[int, str]

Separates repeat factor and json string from string. Separator is * or : If no repeat factor, return just json string. For example:

“3*7” -> 3, [[7]] “3” -> -1, [[3]]

Parameters

jso – string to check

Returns

repeat factor and json-str that stands for a list

timApp.util.secret module

timApp.util.secret.check_secret(secret: str, config_key: str) → None

timApp.util.secret.get_secret_or_abort(config_key: str) → str

timApp.util.testing module

timApp.util.testing.register_testing_routes(app: flask.app.Flask) → None

timApp.util.timtiming module

Functions for dealing taking time.

timApp.util.timtiming.taketime(s1: str = '', s2: str = '', n: int = 0, zero: bool = False) → None

timApp.util.timtiming.with_timing(*print_args: str) → Callable

A decorator for printing timing info for a function. Usage example:

@with_timing('name')
def f(name, some_other_param):
    ...

Parameters

print_args – Names of parameters whose values should be printed to help differentiate calls.

timApp.util.utils module

Utility functions.

timApp.util.utils.append_to_bytearray(b: bytearray, v: Any) → None

timApp.util.utils.approximate_real_name(email: str) → str

class timApp.util.utils.cached_property(func: Callable)

Bases: object

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property. Source: https://github.com/bottlepy/bottle/commit/fa7733e075da0d790d809aa3d2f53071897e6f76

timApp.util.utils.collect_errors_from_hosts(futures: list[concurrent.futures._base.Future], hosts: list[str]) → list[str]

timApp.util.utils.convert_email_to_lower(email_or_username: str) → str

timApp.util.utils.count_chars_from_beginning(md: str, char: str) → int

timApp.util.utils.dataclass_to_bytearray(x: Any) → bytearray

timApp.util.utils.date_to_relative(d: datetime.datetime) → str

Converts the given datetime object to relative string representation, such as “2 days ago”, etc.

Parameters

d – The datetime object to convert.

Returns

A string representing the given date relative to current time.

timApp.util.utils.datestr_to_relative(d: str | datetime.datetime) → str

timApp.util.utils.decode_csplugin(text: lxml.html.HtmlElement) → dict[str, Any]

timApp.util.utils.del_content(directory: pathlib.Path, onerror: Optional[Callable[[Any, str, Any], Any]] = None) → None

timApp.util.utils.exclude_keys(obj: dict[str, Any], *keys: str) → dict[str, Any]

timApp.util.utils.get_alias(name: str) → str

Get name part form string like 534.d1.points

Parameters

name – full name of field

Returns

just name part of field, like d1

timApp.util.utils.get_boolean(s: bool | int | str, default: bool) → bool

timApp.util.utils.get_current_time() → datetime.datetime

timApp.util.utils.get_dataclass_field_names(d: Any) → set[str]

timApp.util.utils.get_error_html(message: str | Exception, response: Optional[str] = None) → str

Wraps an error message in an HTML element with class ‘error’.

Parameters

• response – The plugin response string.

• message – The message to be displayed in the error.

Returns

The sanitized error message HTML.

timApp.util.utils.get_error_html_block(title: str, message: str | Exception, response: Optional[str] = None) → str

Wraps an error message in a block-level HTML element with class ‘error’.

Parameters

• title – Title of the error.

• response – The plugin response string.

• message – The message to be displayed in the error.

Returns

The sanitized error message HTML.

timApp.util.utils.get_error_message(e: Exception) → str

Gives error message with error class. :param e: Exception. :return: String ‘ErrorClass: reason’.

timApp.util.utils.get_error_tex(title: str, message: str | Exception) → str

Wraps an error message in a TeX element ‘timpluginerror’.

Parameters

• title – The plugin response string.

• message – The message to be displayed in the error.

Returns

The sanitized error message HTML.

timApp.util.utils.get_exception_code(ex: Exception, tb: Optional[types.TracebackType] = None) → str

Creates a unique code for the exception and the optional traceback. The code can be used for short identification of errors.

Parameters

• ex – Exception object

• tb – Optional traceback object

Returns

A string code of format ExceptionType[_HexCode] where _HexCode is generated from the traceback

timApp.util.utils.get_sql_template(value_list: list) → str

timApp.util.utils.get_static_tim_doc_path() → pathlib.Path

timApp.util.utils.getdatetime(s: str, default_val: Optional[datetime.datetime] = None) → datetime.datetime | None

timApp.util.utils.include_keys(obj: dict[str, Any], *keys: str) → dict[str, Any]

timApp.util.utils.is_valid_email(email: str) → bool

timApp.util.utils.join_location(location: str, name: str) → str

timApp.util.utils.partition(pred: Callable[[timApp.util.utils.TItem], bool], iterable: Iterable[timApp.util.utils.TItem]) → tuple[Iterable[TItem], Iterable[TItem]]

Use a predicate to partition entries into false entries and true entries.

Parameters

• pred – Predicate to determine in which tuple entries will be placed.

• iterable – Iterable that is divided into a tuple based on the predicate.

Returns

True entries go to the left, and false entries to the right.

timApp.util.utils.pycharm_running() → bool

timApp.util.utils.read_json_lines(file_to_read: pathlib.Path) → list[dict]

timApp.util.utils.relative_location(location: str, base: str) → str

Returns the location of location relative to base.

timApp.util.utils.remove_path_special_chars(item_path: str) → str

timApp.util.utils.remove_prefix(text: str, prefix: str) → str

timApp.util.utils.render_raw_template_string(template: str, **context: Any) → str

Renders a Jinja2 template outside Flask’s context. Skips any injected variables.

Parameters

• template – Template to render.

• context – Specific values to inject.

Returns

Rendered template.

timApp.util.utils.seq_to_str(lst: Sequence[str]) → str

timApp.util.utils.split_by_semicolon(p: str) → list[str]

timApp.util.utils.split_location(path: str) → tuple[str, str]

Given a path ‘a/b/c/d’, returns a tuple (‘a/b/c’, ‘d’).

timApp.util.utils.static_tim_doc(path: str) → str

timApp.util.utils.title_to_id(s: str) → str

Converts a HTML heading to id attribute. Tries to be equivalent to what Pandoc does.

timApp.util.utils.trim_markdown(text: str) → str

Trims the specified text. Don’t trim spaces from left side because they may indicate a code block.

Parameters

text – The text to be trimmed.

Returns

The trimmed text.

timApp.util.utils.try_load_json(json_str: str | None) → Union[None, str, Any]

timApp.util.utils.wait_response_and_collect_error(f: concurrent.futures._base.Future, h: str, errors: list[str]) → None

timApp.util.utils.widen_fields(fields: list[str] | str) → list[str]

if there is syntax d(1,3) in fileds, it is made d1,d2 from d(1,3)=t would come d1=t1, d2=t2

Parameters

fields – list of fields

Returns

array fields widened

Module contents