Feature: add changes from 1and1 branch, rebase + resolve conflicts for current main by erickloss · Pull Request #42 · Flowpack/Flowpack.DecoupledContentStore

erickloss · 2025-04-23T10:05:42Z

No description provided.

With this change the tailwind styles are only applied to the app container of the decoupled content store

This stops having to go through all individual worker outputs

…not available

…adata

Each document needs to be checked whether it’s still connected to the page tree though.

Only upload script once and reuse script and redis instance

BUGFIX: Add html escaping to stderr and stdout of job logs

…ack/Flowpack.DecoupledContentStore into task/adjustments-for-1and1

Previously if `ContentReleaseBatchResult` contained a wrong value or the fallback null the view would crash.

Related: #35

…ce-and-highlight-error BUGFIX: Joblogs preserve whitespace and highlight errors

If a line contains json output it is rendered as toggable details element

skurfuerst

Hey everybody, specifically @Sebobo and @erickloss ,

thanks for your contributions ❤️ Sorry I only now saw this pull request, please ping me next time in Slack :)

I am currently working on a bigger overhaul of the Decoupled ContentStore package, which will support other rendering targets than the default Fusion rendering as well. In the course of doing so, I have reviewed this change, and would like to discuss a few details. Overally, the code looks great ❤️ ❤️ ❤️

I'd love to make this a part of version 2 (my changes will be a bit breaking config-wise, but easy to adjust).

All the best,
Sebastian
PS: I am on holidays the next two weeks, so we'll find a spot for discussing this afterwards :)

skurfuerst · 2025-07-16T18:02:45Z

Classes/Core/ConcurrentBuildLockService.php


 /**
- * We usually rely on prunner to ensure that only one build is running at any given time.
+ * We usually rely on prunner to ensure that only one build per workspace is running at any given time.


new feature (for changelog / upgrade notes)

skurfuerst · 2025-07-16T18:03:04Z

Classes/Eel/ModuleHelper.php

+            // Remove additional JSON data from log line
+            $line = preg_replace("/\{.*}/", '', $line);
+
+            // Add highlighting for log levels


new feature (for changelog / upgrade notes)

skurfuerst · 2025-07-16T18:03:13Z

Classes/IncrementalContentReleaseHandler.php

    protected $contentReleaseManager;

+    /**
+     * @Flow\InjectConfiguration("startIncrementalReleaseOnWorkspacePublish")


new feature (for changelog / upgrade notes)

skurfuerst · 2025-07-16T18:04:24Z

Classes/NodeEnumeration/Domain/Dto/EnumeratedNode.php

        return [
            'contextPath' => $this->contextPath,
            'nodeIdentifier' => $this->nodeIdentifier,
+            'dimensions' => $this->getDimensionsFromContextPath(),


why dimensions extra here?

I'm not sure anymore. Probably unused.

skurfuerst · 2025-07-16T18:04:41Z

Classes/NodeEnumeration/Domain/Service/NodeContextCombinator.php

    protected $contextFactory;

+    /**
+     * @Flow\InjectConfiguration(path="nodeRendering.recurseHiddenContent", package="Flowpack.DecoupledContentStore")


new feature (for changelog / upgrade notes)

skurfuerst · 2025-07-16T18:06:57Z

Classes/NodeEnumeration/NodeEnumerator.php


-                        $contentReleaseLogger->debug('Registering node for publishing', [
-                            'node' => $contextPath
+                    if ($nodeToEnumerate->isHidden()) {


I thought you wanted to allow hidden nodes as well?

We allow hidden content (due to project specifics), but not hidden documents.

skurfuerst · 2025-07-16T18:07:40Z

Classes/NodeRendering/Dto/DocumentNodeCacheKey.php

    public static function fromEnumeratedNode(EnumeratedNode $enumeratedNode)
    {
-        return new self($enumeratedNode->getNodeIdentifier(), $enumeratedNode->getDimensionsFromContextPath(), $enumeratedNode->getArguments());
+        return new self($enumeratedNode->getNodeIdentifier(), $enumeratedNode->getDimensionsFromContextPath(), $enumeratedNode->getWorkspaceNameFromContextPath(), $enumeratedNode->getArguments());


how does this change do anything? (modifications in this class)

This was not fully finished, see TODO below. I think we were lucky so far, that we didn't have collisions due to this.

skurfuerst · 2025-07-16T18:08:16Z

Classes/NodeRendering/Infrastructure/RedisContentCacheReader.php

+            'The cache backend for "Neos_Fusion_Content" must be an OptimizedRedisCacheBackend, but is ' . get_class(
+                $backend
+            ), 1622570000
+        );


I believe this is only code refactoring, no functional change?

No it was an optimisation Only upload script once and reuse script and redis instance

skurfuerst · 2025-07-16T18:09:16Z

Classes/PrepareContentRelease/Dto/ContentReleaseMetadata.php

                return PrunnerJobId::fromString($item);
            }, json_decode($tmp['manualTransferJobIds'])) : [],
-            $tmp['workspace'] ?? 'live',
+            $tmp['workspaceName'] ?? 'live',


TODO check if this is correct

Yes, see the jsonSerialize method of ContentReleaseMetadata

skurfuerst · 2025-07-16T18:11:19Z

pipelines_template.yml

        script:
          - ./flow contentReleasePrepare:createContentRelease {{ .contentReleaseId }} {{ .__jobID }} --workspaceName {{ .workspaceName }} --accountId {{ .accountId }}
          - ./flow contentReleasePrepare:ensureAllOtherInProgressContentReleasesWillBeTerminated {{ .contentReleaseId }}
+          - ./flow contentReleasePrepare:flushContentCacheIfRequired {{ .contentReleaseId }} --flushContentCache={{ .flushContentCache }}


BREAKING, so needs tobe added to own pipelines.yaml when updating to 2.0

Sebobo and others added 30 commits January 19, 2023 14:51

BUGFIX: Show module icon in module list

b77c2b7

BUGFIX: Prevent tailwind from breaking neos backend styles

fb3ee84

With this change the tailwind styles are only applied to the app container of the decoupled content store

BUGFIX: Prevent metadata from loosing its workspacename

5e53d53

FEATURE: Set build lock by workspace to allow parallel releases

6b25d92

TASK: Show rendering error details in render_orchestrator output

8b5f110

This stops having to go through all individual worker outputs

BUGFIX: Flush content cache when the pipeline actually starts

d39ef93

FEATURE: Allow recursing through hidden documents

28f7d2f

add configuration to enable/disable auto-start of incremental releases

efc5811

BUGFIX: Use workspace name from enumerated node during rendering

eb2bca8

BUGFIX: Enable storing rendered documents in non public workspaces

b0aabd7

BUGFIX: Prevent exception in overview when ContentReleaseMetadata is …

a0de60d

…not available

TASK: Add rendertime to render orchestrator output

477108a

BUGFIX: Prevent error when exception has no referenceCode

4876ca0

BUGFIX: Prevent creating a second content release with the same id

f3f7110

FEATURE: Store rendertime of documents in metadata

7ce1709

BUGFIX: Prevent error when viewing content releases with no Redis met…

8bafbae

…adata

FEATURE: Log enumeration duration for each site

a15d22e

BUGFIX: Speed up enumeration by fetching all documents with one query

8e20be9

Each document needs to be checked whether it’s still connected to the page tree though.

TASK: Cache redis instance for repeated calls in CacheReader

9e988aa

FEATURE: Allow enumeration of any nodetype

3816e27

TASK: Optimise redis content cache reader

a59129f

Only upload script once and reuse script and redis instance

TASK: Cleanup enumeration code

b07abbc

TASK: Reformat NodeEnumerator code

ba2248f

BUGFIX: Add html escaping to stderr and stdout of job logs

74157bf

Merge pull request #35 from Flowpack/bugfix/error-logs-html-escaping

a737e87

BUGFIX: Add html escaping to stderr and stdout of job logs

FEATURE: Make iteration checks in NodeRenderer configurable

ce64170

Merge branch 'task/adjustments-for-1and1' of https://github.com/Flowp…

cb4128d

…ack/Flowpack.DecoupledContentStore into task/adjustments-for-1and1

TASK: Append payload only in logging if set

dbb3705

TASK: Add newlines to log output

9e887c9

BUGFIX: Remove pretty formatting for payloads again

a639800

Sebobo and others added 8 commits December 12, 2023 16:03

BUGFIX: Make NodeRenderer constants overridable

f68938e

TASK: Forgiving data handling in ui service

53db941

Previously if `ContentReleaseBatchResult` contained a wrong value or the fallback null the view would crash.

BUGFIX: Joblogs preserve whitespace and highlight errors

aab5a6d

Related: #35

TASK: Reverse stdout log and improve styling

11a2a2b

Merge pull request #36 from Flowpack/bugfix/joblogs-preserve-whitespa…

71a4ac5

…ce-and-highlight-error BUGFIX: Joblogs preserve whitespace and highlight errors

FEATURE: Format stdout with eel helper

d62e013

If a line contains json output it is rendered as toggable details element

BUGFIX: Handle empty stdout when formatting logs

7d576fe

Merge branch 'main' into task/adjustments-for-1and1

0c46d50

erickloss requested review from JamesAlias and Sebobo April 23, 2025 10:06

erickloss added 3 commits May 8, 2025 14:38

BUGFIX: site nodes do not get recognized as orphaned anymore

44680cb

BUGFIX: merge error

53c26b3

BUGFIX: merge error

b0ca14d

skurfuerst requested changes Jul 16, 2025

View reviewed changes

Conversation

erickloss commented Apr 23, 2025

Uh oh!

skurfuerst left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants