keychron_qmk_firmware/lib/python/qmk/commands.py

236 lines
6.7 KiB
Python
Raw Normal View History

"""Helper functions for commands.
"""
import os
import sys
import json
import shutil
from pathlib import Path
from milc import cli
import jsonschema
2022-03-18 22:02:24 +06:00
from qmk.constants import KEYBOARD_OUTPUT_PREFIX
from qmk.json_schema import json_load, validate
def _find_make():
"""Returns the correct make command for this environment.
"""
make_cmd = os.environ.get('MAKE')
if not make_cmd:
make_cmd = 'gmake' if shutil.which('gmake') else 'make'
return make_cmd
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
def create_make_target(target, dry_run=False, parallel=1, **env_vars):
"""Create a make command
Args:
target
Usually a make rule, such as 'clean' or 'all'.
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
dry_run
make -n -- don't actually build
parallel
The number of make jobs to run in parallel
**env_vars
Environment variables to be passed to make.
Returns:
A command that can be run to make the specified keyboard and keymap
"""
env = []
make_cmd = _find_make()
for key, value in env_vars.items():
env.append(f'{key}={value}')
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
return [make_cmd, *(['-n'] if dry_run else []), *get_make_parallel_args(parallel), *env, target]
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
def create_make_command(keyboard, keymap, target=None, dry_run=False, parallel=1, **env_vars):
"""Create a make compile command
Args:
keyboard
The path of the keyboard, for example 'plank'
keymap
The name of the keymap, for example 'algernon'
target
Usually a bootloader.
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
dry_run
make -n -- don't actually build
parallel
The number of make jobs to run in parallel
**env_vars
Environment variables to be passed to make.
Returns:
A command that can be run to make the specified keyboard and keymap
"""
make_args = [keyboard, keymap]
if target:
make_args.append(target)
New CLI subcommand to create clang-compatible compilation database (`compile_commands.json`) (#14370) * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * pulled source from dev branch * missed a file from origin * formatting * revised argument names. relaxed matching rules to work for avr too * add docstrings * added docs. tightened up regex * remove unused imports * cleaning up command file. use existing qmk dir constant * rename parser library file * move lib functions into command file. there are only 2 and they aren't large * currently debugging... * more robustly find config * updated docs * remove unused imports * reuse make executable from the main make command * remove MAKEFLAGS from environment for better control over process management * Update .gitignore Co-authored-by: Michael Forster <forster@google.com> * add a usage line to docs * doc change as suggested Co-authored-by: Nick Brassel <nick@tzarc.org> * rename command * remove debug print statements * generate-compilation-database: fix arg handling * generate-comilation-db: improve error handling * use cli.run() instead of Popen() Co-authored-by: Xton <cdewan@apple.com> Co-authored-by: Christon DeWan <cmdpix@mac.com> Co-authored-by: Michael Forster <forster@google.com> Co-authored-by: Nick Brassel <nick@tzarc.org>
2021-09-16 10:59:57 +06:00
return create_make_target(':'.join(make_args), dry_run=dry_run, parallel=parallel, **env_vars)
def get_make_parallel_args(parallel=1):
"""Returns the arguments for running the specified number of parallel jobs.
"""
parallel_args = []
if int(parallel) <= 0:
# 0 or -1 means -j without argument (unlimited jobs)
parallel_args.append('--jobs')
else:
parallel_args.append('--jobs=' + str(parallel))
if int(parallel) != 1:
# If more than 1 job is used, synchronize parallel output by target
parallel_args.append('--output-sync=target')
return parallel_args
2021-02-02 01:55:35 +06:00
def compile_configurator_json(user_keymap, bootloader=None, parallel=1, **env_vars):
"""Convert a configurator export JSON file into a C file and then compile it.
Args:
user_keymap
A deserialized keymap export
bootloader
A bootloader to flash
parallel
The number of make jobs to run in parallel
Returns:
A command to run to compile and flash the C file.
"""
# In case the user passes a keymap.json from a keymap directory directly to the CLI.
# e.g.: qmk compile - < keyboards/clueboard/california/keymaps/default/keymap.json
user_keymap["keymap"] = user_keymap.get("keymap", "default_json")
# Write the keymap.c file
keyboard_filesafe = user_keymap['keyboard'].replace('/', '_')
target = f'{keyboard_filesafe}_{user_keymap["keymap"]}'
keyboard_output = Path(f'{KEYBOARD_OUTPUT_PREFIX}{keyboard_filesafe}')
keymap_output = Path(f'{keyboard_output}_{user_keymap["keymap"]}')
keymap_dir = keymap_output / 'src'
keymap_json = keymap_dir / 'keymap.json'
keymap_dir.mkdir(exist_ok=True, parents=True)
keymap_json.write_text(json.dumps(user_keymap), encoding='utf-8')
# Return a command that can be run to make the keymap and flash if given
verbose = 'true' if cli.config.general.verbose else 'false'
color = 'true' if cli.config.general.color else 'false'
make_command = [_find_make()]
if not cli.config.general.verbose:
make_command.append('-s')
make_command.extend([
*get_make_parallel_args(parallel),
'-r',
'-R',
'-f',
'builddefs/build_keyboard.mk',
])
2021-02-02 01:55:35 +06:00
if bootloader:
make_command.append(bootloader)
for key, value in env_vars.items():
make_command.append(f'{key}={value}')
make_command.extend([
f'KEYBOARD={user_keymap["keyboard"]}',
f'KEYMAP={user_keymap["keymap"]}',
f'KEYBOARD_FILESAFE={keyboard_filesafe}',
f'TARGET={target}',
f'KEYBOARD_OUTPUT={keyboard_output}',
f'KEYMAP_OUTPUT={keymap_output}',
f'MAIN_KEYMAP_PATH_1={keymap_output}',
f'MAIN_KEYMAP_PATH_2={keymap_output}',
f'MAIN_KEYMAP_PATH_3={keymap_output}',
f'MAIN_KEYMAP_PATH_4={keymap_output}',
f'MAIN_KEYMAP_PATH_5={keymap_output}',
f'KEYMAP_JSON={keymap_json}',
f'KEYMAP_PATH={keymap_dir}',
f'VERBOSE={verbose}',
f'COLOR={color}',
'SILENT=false',
'QMK_BIN="qmk"',
])
return make_command
def parse_configurator_json(configurator_file):
"""Open and parse a configurator json export
"""
user_keymap = json_load(configurator_file)
# Validate against the jsonschema
try:
validate(user_keymap, 'qmk.keymap.v1')
except jsonschema.ValidationError as e:
cli.log.error(f'Invalid JSON keymap: {configurator_file} : {e.message}')
exit(1)
orig_keyboard = user_keymap['keyboard']
aliases = json_load(Path('data/mappings/keyboard_aliases.json'))
if orig_keyboard in aliases:
if 'target' in aliases[orig_keyboard]:
user_keymap['keyboard'] = aliases[orig_keyboard]['target']
if 'layouts' in aliases[orig_keyboard] and user_keymap['layout'] in aliases[orig_keyboard]['layouts']:
user_keymap['layout'] = aliases[orig_keyboard]['layouts'][user_keymap['layout']]
return user_keymap
def in_virtualenv():
"""Check if running inside a virtualenv.
Based on https://stackoverflow.com/a/1883251
"""
active_prefix = getattr(sys, "base_prefix", None) or getattr(sys, "real_prefix", None) or sys.prefix
return active_prefix != sys.prefix
def dump_lines(output_file, lines, quiet=True):
"""Handle dumping to stdout or file
Creates parent folders if required
"""
generated = '\n'.join(lines) + '\n'
if output_file and output_file.name != '-':
output_file.parent.mkdir(parents=True, exist_ok=True)
if output_file.exists():
output_file.replace(output_file.parent / (output_file.name + '.bak'))
output_file.write_text(generated, encoding='utf-8')
if not quiet:
cli.log.info(f'Wrote {output_file.name} to {output_file}.')
else:
print(generated)