metapython.parse

metapython.parse.string_from_tokens(toks, inline=False)

Build a string from a sequence of tokens. This is a “better” version of tokenize.untokenize, as it verifies that INDENT and DEDENT tokens are appropriately converted to valid Python code (which tokenize.untokenize can fail to do correctly.

metapython.parse.tokens_from_string(s)

Convert a string to a stream of tokens

metapython.parse.tokens_from_file(fp)

Convert a file to a stream of tokens.

metapython.parse.parse_file(fn, namespace=None)

Convert a file to a Block of statements.

metapython.parse.parse_string(s, filename='<string>', inline=False)

Convert a string to a Block of statements.

metapython.parse.parse_stream(tokenstream, filename='<string>', inline=False)

Convert a stream of tokens into a Block of statements.

metapython.parse.expand_macros(tokenstream, glbls=None, lcls=None, filename=None)

Expand $-escapes in a token stream

metapython.parse.expand_inline_codequotes(inp, filename='<string>')

Expand ?-escapes in a token stream

class metapython.parse.Builder

Bases: object

Helper class for building up blocks of code.

A builder maintains a stack of Block objects called the statement stack. This stack can be pushed & popped, and the top of the stack can have new statements appended to it.

append(stmt, glbls, lcls)

Append a statement to the current top of the statement stack

append_suite(header, glbls, lcls)

Create a suite based on the header line given and the block on the top of the stack (which will be popped and used as the body). Then append the newly created suite to the new top of the statement stack.

pop()

Pop and return the block on top of the statement stack.

push()

Push an empty block onto the statement stack.

q(code, inline=True)

Quote a code fragment and parse it.

top

Return the top of the statement stack

class metapython.parse.Stmt(tokens)

Bases: object

Base statement class

append(token)

as_python(inline=False)

expand_defcode_blocks()

first()

quote()

class metapython.parse.Suite(header, body, prologue=None, epilogue=None)

Bases: metapython.parse.Stmt

A block-type statement, with a header and a body

append(token)

as_python(inline=False)

expand_defcode_blocks()

first()

quote()

class metapython.parse.Block(statements=None)

Bases: object

A sequence of statements

append(statement, glbls, lcls)

as_python(inline=False)

eval(glbls, lcls)

exec_(glbls, lcls)

expand(glbls=None, lcls=None)

expand_defcode_blocks()

indent()

one()

quote(code_name=None)

Expand the code in the block under the assumption that it is part of a defcode: block

replace_names(**newnames)

Replace the NAMEs in this block according to the newnames mapping.

sanitize(glbls, lcls, *omit_names)

Ask this block’s builder to sanitize this block, omitting the listed names

class metapython.parse.Token

Bases: tuple

Thin wrapper around tuple to allow access to the token.generate_tokens results by name

assertMatch(tok, *values, **kw)

begin

itemgetter(item, ...) –> itemgetter object

Return a callable object that fetches the given item(s) from its operand. After, f=itemgetter(2), the call f(r) returns r[2]. After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])

end

itemgetter(item, ...) –> itemgetter object

Return a callable object that fetches the given item(s) from its operand. After, f=itemgetter(2), the call f(r) returns r[2]. After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])

indent()

line

itemgetter(item, ...) –> itemgetter object

Return a callable object that fetches the given item(s) from its operand. After, f=itemgetter(2), the call f(r) returns r[2]. After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])

classmethod make(klass, token, value, begin, end, line)

match(tok, *values)

replace(**kwds)

tok_name

token

itemgetter(item, ...) –> itemgetter object

Return a callable object that fetches the given item(s) from its operand. After, f=itemgetter(2), the call f(r) returns r[2]. After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])

value

itemgetter(item, ...) –> itemgetter object

Return a callable object that fetches the given item(s) from its operand. After, f=itemgetter(2), the call f(r) returns r[2]. After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])