18
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-21 21:24:58 +01:00
Files
maszyna/scripting/PyInt.h
maj00r 79c88c1793 Python screens: the worker's image goes in place only after its fence, not a bare glFlush (#411)
* fix(python): a screen uploaded by the worker goes into a texture of its own, put in place after a fence

With python.threadedupload (the default) the worker wrote the image into the texture the drawing thread
draws from, with glTexImage2D and glGenerateMipmap from its own context, and only a glFlush after. Nothing
ordered the two contexts: the drawing one could sample the texture while its storage was being made anew,
its level 0 half written or its mipmaps not made yet, and on a change of size the worker deleted the texture
and wrote a new name into `id` with no synchronisation.

The worker now uploads each image into a new texture, sets a fence after it and flushes. The drawing thread,
once per frame in python_taskqueue::update(), waits for that fence on the gpu (glWaitSync, the cpu goes on),
deletes the old texture and puts the new one in place. The worker no longer touches the texture drawn from,
nor `id`. An image the drawing thread didn't take yet is dropped by the next one. Without sync objects the
worker waits for the upload with glFinish.

The windows of python.displaywindows with python.sharectx bound the texture once; they now bind the name of
the last image put in place, which python_rt keeps for them.

Uploads on the drawing thread (python.threadedupload no, the Lua and WebAssembly screens, the editor) are
as they were.

* fix(python): the image of the worker goes in place once its fence is signalled, the viewer binds it each frame

From the review of 8811ee5a:

- glWaitSync ordered only the context of the main window. The extra viewports and the windows of
  python.displaywindows draw in contexts of their own and could bind the new texture before the upload
  was done. commit_thread_upload() now asks the fence without waiting (glClientWaitSync, timeout 0) and
  puts the image in place only once it is signalled, so every context binding it afterwards has it whole.
  Until then the task stays in the upload queue and is asked again the next frame.
- The drawing thread chose between upload and commit by python.threadedupload, the worker by whether it
  got a context. With python.threadedupload on and no context made for the worker, the screens were no
  longer updated. The task now records that a worker uploaded it.
- The viewer of python.displaywindows told a new image by its texture name, but a deleted name comes back
  for a later texture. It binds the name of the last image each frame, read under the lock of the render
  target, and the name it starts with is read on the drawing thread when it is made.

* fix(python): the old texture deleted under the lock the viewers bind under, a fence not made waited for with glFinish

From the review of #411:

- finish_upload() deleted the old texture before it took the lock of the render target, and the
  windows of python.displaywindows read the name under that lock but bound it after letting go of it.
  A window could bind a name already deleted, or already given to the next upload. In a compatibility
  context (the legacy renderer) that makes a new empty texture: a black frame and a texture never
  deleted. The commit and the publishing of the new name now go under the lock, and the viewer binds
  under it too. A name bound before the deletion stays alive in that window's context while bound.
- A glFenceSync that returns no fence was taken for a signalled one: the image went in place at once,
  with nothing ordering the contexts. The worker now waits for its upload with glFinish then, as
  without sync objects.

* fix(python): the worker waits for the lock of the screen without the gil, no flush after a finish

From the review of 939514ea:

- The drawing thread now holds the lock of the render target for a whole upload with
  python.threadedupload off. The worker, copying the next image under that lock, waited for it holding
  the gil. It now lets the gil go around the lock and the copy: the bytes it copies stay valid, the
  object is immutable and held by the reference the worker has.
- glFlush after glFinish did nothing; the worker flushes only when it made a fence.

---------

Co-authored-by: maj00r <maj00r@host>
2026-11-20 03:39:42 +01:00

169 lines
4.5 KiB
C++

/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#ifndef PYINT_H
#define PYINT_H
#ifdef _POSIX_C_SOURCE
#undef _POSIX_C_SOURCE
#endif
#ifdef _XOPEN_SOURCE
#undef _XOPEN_SOURCE
#endif
#ifdef _MSC_VER
#pragma warning(push)
#pragma warning(disable : 5033)
#endif
#ifdef __GNUC__
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wregister"
#endif
#ifdef WITH_PYTHON
#ifdef _DEBUG
#undef _DEBUG // bez tego macra Py_DECREF powoduja problemy przy linkowaniu
#include "Python.h"
#define _DEBUG
#else
#include "Python.h"
#endif
#else
#define PyObject void
#define PyThreadState void
#endif
#ifdef __GNUC__
#pragma GCC diagnostic pop
#endif
#ifdef _MSC_VER
#pragma warning(pop)
#endif
#include "utilities/Classes.h"
#include "utilities/utilities.h"
#include "model/Texture.h"
#include <thread>
#define PyGetFloat(param) PyFloat_FromDouble(param)
#define PyGetBool(param) param ? Py_True : Py_False
// python rendertarget
struct python_rt
{
std::mutex mutex;
ITexture *shared_tex;
int format;
int components;
int width;
int height;
std::string image;
// gl name of the texture with the last image put in place, for the windows of python.displaywindows (0: none yet). the image
// uploaded by a worker gets a texture of its own, so the name changes with each one
size_t texture_id{0};
std::chrono::high_resolution_clock::time_point timestamp;
};
// TODO: extract common base and inherit specialization from it
class render_task
{
public:
// constructors
render_task(PyObject *Renderer, std::shared_ptr<dictionary_source> Input, std::shared_ptr<python_rt> Target) : m_renderer(Renderer), m_input(Input), m_target(Target) {}
// methods
void run();
// uploads the image on the drawing thread
void upload() const;
// uploads the image on a worker with a gl context of its own (python.threadedupload); finish_upload() puts it in place
void upload_from_thread();
// on the drawing thread: uploads the image, or puts in place the one a worker uploaded. false while the gpu hasn't got
// the latter yet, to be called again
bool finish_upload() const;
void cancel() const;
auto target() const -> std::shared_ptr<python_rt>
{
return m_target;
}
private:
// members
PyObject *m_renderer{nullptr};
std::shared_ptr<dictionary_source> m_input{nullptr};
std::shared_ptr<python_rt> m_target{nullptr};
// set by upload_from_thread(); read by the drawing thread after the task was handed over under the lock of the queue
bool m_uploadedbyworker{false};
};
class python_taskqueue
{
public:
// types
struct task_request
{
std::string const &renderer;
std::shared_ptr<dictionary_source> input;
std::shared_ptr<python_rt> target;
};
// constructors
python_taskqueue() = default;
// methods
// initializes the module. returns true on success
auto init() -> bool;
// shuts down the module
void exit();
// adds specified task along with provided collection of data to the work queue. returns true on success
auto insert(task_request const &Task) -> bool;
// executes python script stored in specified file. returns true on success
auto run_file(std::string const &File, std::string const &Path = "") -> bool;
// acquires the python gil and sets the main thread as current
void acquire_lock();
// releases the python gil and swaps the main thread out
void release_lock() const;
void update();
private:
// types
static int const WORKERCOUNT{1};
using worker_array = std::array<std::jthread, WORKERCOUNT>;
using rendertask_sequence = threading::lockable<std::deque<std::shared_ptr<render_task>>>;
using uploadtask_sequence = threading::lockable<std::deque<std::shared_ptr<render_task>>>;
// methods
auto fetch_renderer(std::string const &Renderer) -> PyObject *;
void run(GLFWwindow *Context, rendertask_sequence &Tasks, uploadtask_sequence &Upload_Tasks, threading::condition_variable &Condition, std::atomic<bool> const &Exit);
void error();
// members
PyObject *m_main{nullptr};
PyObject *m_stderr{nullptr};
PyThreadState *m_mainthread{nullptr};
worker_array m_workers;
threading::condition_variable m_condition; // wakes up the workers
std::atomic<bool> m_exit{false}; // signals the workers to quit
std::unordered_map<std::string, PyObject *> m_renderers; // cache of python classes
rendertask_sequence m_tasks;
uploadtask_sequence m_uploadtasks;
bool m_initialized{false};
};
class python_external_utils
{
public:
static std::vector<std::string> PyObjectToStringArray(PyObject *pyList);
};
#endif