Skip to content

CI: use EngFlow's opal cluster for remote execution#210

Draft
jayconrod wants to merge 5 commits intomainfrom
remote-execution
Draft

CI: use EngFlow's opal cluster for remote execution#210
jayconrod wants to merge 5 commits intomainfrom
remote-execution

Conversation

@jayconrod
Copy link
Copy Markdown
Collaborator

Platform configurations and most flags copied from
github.com/EngFlow/example.

Fixes #31

Signed-off-by: Jay Conrod jay@engflow.com

Platform configurations and most flags copied from
github.com/EngFlow/example.

Fixes #31

Signed-off-by: Jay Conrod <jay@engflow.com>
Signed-off-by: Jay Conrod <jay@engflow.com>
Signed-off-by: Jay Conrod <jay@engflow.com>
Signed-off-by: Jay Conrod <jay@engflow.com>
Signed-off-by: Jay Conrod <jay@engflow.com>
@jayconrod
Copy link
Copy Markdown
Collaborator Author

cc @mateusz-olczyk FYI, this is a work in progress. Not working yet.

  • The macOS build job is being picked up and is executing actions remotely, but is failing due to a permissions error in prepare_test_repo.py. Might be a remote execution readiness problem. The error message is shown in the console log.
  • The bcr_test_module_macos job is failing due to an authentication issue. Probably a misconfiguration here on my parent.
  • Linux jobs are not being picked up. These use a different type of CI runner, and the server needs to be configured to poll jobs from this repo. Not sure how to do this, I need to work with the internal team. This is an unusual configuration; as far as I can tell, other public repos that do this are misconfigured, so I don't want to copy them.

@mateusz-olczyk
Copy link
Copy Markdown
Collaborator

  • The macOS build job is being picked up and is executing actions remotely, but is failing due to a permissions error in prepare_test_repo.py. Might be a remote execution readiness problem. The error message is shown in the console log.

The problem might come from here:

if file == BUILD_FILE_NAME:
    # If this is a BUILD.bazel file (after renaming), append a filegroup with the collected rules
    dest_file = out_root / "BUILD.bazel"
    shutil.copy2(src_file, dest_file)
    if rule_names := parse_rule_names(src_file.read_text()):
        append_filegroup(dest_file, rule_names)
        filegroup_labels.add(filegroup_label)

I think shutil.copy2 preserves the attributes of src_file. But in this if branch, we need to open the file in "append" mode. If the runner checks out files from the repo in read-only mode, then the code's logic is incorrect. You might want to change shutil.copy2 with something, which ensures "write" access.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run CI on CI runners

2 participants