franklinblanco/Blender_bevy_components_workflow
1
0
Fork 0
mirror of https://github.com/kaosat-dev/Blender_bevy_components_workflow.git synced 2024-11-22 11:50:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(Blenvy:Blender): fixes & enhancements to auto export, particularly assets

Browse Source 
* injected BlueprintAssets are now reusing existing asset scan boilerplate (wip)
 * added back per blueprint assets
 * experimenting with how to export all vs local only assets
 * renamed (finally) export main scenes
 * found issue with scene serialization (hellooo collections !), working on fix
This commit is contained in:
kaosat.dev 2024-07-11 01:06:27 +02:00
parent 33cddda7a5
commit 270202d24f
7 changed files with 165 additions and 92 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

55
tools/blenvy/TODO.md
View File

@ -185,7 +185,37 @@ Blender side:
- [x] BLENVY_OT_item_select is missing handling for the other types (outside of object & collection)
- [x] fix selection logic
- [x] update testing blend files
- [x] disable 'export_hierarchy_full_collections' for all cases: not reliable and redudant
- [ ] fix systematic material exports despite no changes
- [ ] investigate lack of detection of changes of adding/changing components
- [ ] change scene serialization to account for collections ...sigh
- [ ] also remove ____dummy____.bin when export format is gltf
- [ ] fix/cleanup asset information injection (also needed for hot reload)
- [ ] add back per blueprint assets
- [ ] reuse the already existing asset_scan + export thing
- thoughts:
- the "list of all assets" is actually the "fake"/generated one: nobody would write a list of assets for sub assets,
you would just add the assets to your blueprint
- in Bevy at spawning we have
blueprint => assets
for hot reload we need
asset => blueprint instances so we can despawn/respawn etc blueprint instances when one of their assets has changed
problem of untyped vs typed
perhaps have a mapping of untyped => typed id
map asset id => [entity ids]
- [ ] add option to 'split out' meshes from blueprints ?
- [ ] ie considering meshletts etc , it would make sense to keep blueprints seperate from purely mesh gltfs
- [ ] persist exported materials path in blueprints so that it can be read from library file users
- [ ] just like "export_path" write it into each blueprint's collection
- [ ] scan for used materials per blueprint !
- [ ] for scenes, scan for used materials of all non instance objects (TODO: what about overrides ?)
- [ ] add a way of visualizing per blueprint instances ?
- [ ] display export path of blueprints (mostly external) ?
- [ ] hidden objects/collections only semi respected at export
- this is because blueprints are external ?
- [ ] verify based on gltf settings
@ -199,20 +229,8 @@ Blender side:
- [ ] disabled components
- [ ] blueprint instances as children of blueprint instances
- [ ] blueprint instances as children of empties
- [x] update testing blend files
- [ ] disable 'export_hierarchy_full_collections' for all cases: not reliable and redudant
- [ ] add option to 'split out' meshes from blueprints ?
- [ ] ie considering meshletts etc , it would make sense to keep blueprints seperate from purely mesh gltfs
- [ ] persist exported materials path in blueprints so that it can be read from library file users
- [ ] just like "export_path" write it into each blueprint's collection
- [ ] scan for used materials per blueprint !
- [ ] for scenes, scan for used materials of all non instance objects (TODO: what about overrides ?)
- [ ] add a way of visualizing per blueprint instances ?
- [ ] display export path of blueprints (mostly external) ?
Bevy Side:
- [x] deprecate BlueprintName & BlueprintPath & use BlueprintInfo instead
- [x] make blueprint instances invisible until spawning is done to avoid "spawn flash"?
@ -237,17 +255,26 @@ Bevy Side:
- [x] blueprint level/ collection level components are now visible in instances in Blender
- [x] they do not seem to be transfered to the (instance) entity above:
could they be on the "empty node" ?
- [ ] add back & cleanup animation frame triggers
- [ ] simplify testing example:
- [x] remove use of rapier physics (or even the whole common boilerplate ?)
- [ ] remove/replace bevy editor pls with some native ui to display hierarchies
- [ ] a full fledged demo (including physics & co)
- [ ] other examples without interactions or physics
- [ ] add hot reloading
- [x] basics
- [x] make it enabled/disabled based on general flag
- [ ] make
- [ ] cleanup internals
- [x] account for changes impact both parent & children (ie "world" and "blueprint3") for example, which leads to a crash as there is double despawn /respawn so we need to filter things out
- [x] if there are many assets/blueprints that have changed at the same time, it causes issues similar to the above, so apply a similar fix
- [x] also ignore any entities currently spawning (better to loose some information, than cause a crash)
- [ ] something is off with blueprint level components
- [ ] add the root blueprint itself to the assets either on the blender side or on the bevy side programatically
- [x] for sub blueprint tracking: do not propagate/ deal with parent blueprints if they are not themselves Spawning (ie filter out by "BlueprintSpawning")
- [ ] invalidate despawned entity & parent entities AABB
- [x] cleanup internals
- [x] review & change general component insertion & spawning ordering & logic
- GltfComponentsSet::Injection => GltfBlueprintsSet::Spawn => GltfBlueprintsSet::AfterSpawn

25
tools/blenvy/add_ons/auto_export/blueprints/export_blueprints.py
View File

@ -5,41 +5,28 @@ from blenvy.assets.generate_asset_file import write_ron_assets_file
from ..constants import TEMPSCENE_PREFIX
from ..common.generate_temporary_scene_and_export import generate_temporary_scene_and_export, copy_hollowed_collection_into, clear_hollow_scene
from ..common.export_gltf import generate_gltf_export_settings
def assets_to_fake_ron(list_like):
result = []
for item in list_like:
result.append(f"(name: \"{item['name']}\", path: \"{item['path']}\")")
return f"(assets: {result})".replace("'", '')
return f"({result})".replace("'", '')
from ..utils import upsert_blueprint_assets
def export_blueprints(blueprints, settings, blueprints_data):
blueprints_path_full = getattr(settings, "blueprints_path_full")
gltf_export_settings = generate_gltf_export_settings(settings)
export_materials_library = getattr(settings.auto_export, "export_materials_library")
try:
# save current active collection
active_collection = bpy.context.view_layer.active_layer_collection
export_materials_library = getattr(settings.auto_export, "export_materials_library")
for blueprint in blueprints:
print("exporting collection", blueprint.name)
gltf_output_path = os.path.join(blueprints_path_full, blueprint.name) # TODO: reuse the export_path custom property ?
gltf_export_settings = { **gltf_export_settings, 'use_active_scene': True, 'use_active_collection': True, 'use_active_collection_with_nested':True}
collection = bpy.data.collections[blueprint.name]
# if we are using the material library option, do not export materials, use placeholder instead
if export_materials_library:
gltf_export_settings['export_materials'] = 'PLACEHOLDER'
collection = bpy.data.collections[blueprint.name]
all_assets = []
auto_assets = []
collection["BlueprintAssets"] = assets_to_fake_ron([]) #assets_to_fake_ron([{"name": asset.name, "path": asset.path} for asset in collection.user_assets] + auto_assets) #all_assets + [{"name": asset.name, "path": asset.path} for asset in collection.user_assets] + auto_assets)
gltf_export_settings['export_materials'] = 'PLACEHOLDER'
upsert_blueprint_assets(blueprint, blueprints_data=blueprints_data, settings=settings)
# do the actual export
generate_temporary_scene_and_export(

6
tools/blenvy/add_ons/auto_export/common/auto_export.py
View File

@ -10,7 +10,7 @@ from ..blueprints.get_blueprints_to_export import get_blueprints_to_export
from ..levels.get_levels_to_export import get_levels_to_export
from .export_gltf import get_standard_exporter_settings
from ..levels.export_main_scenes import export_main_scene
from ..levels.export_levels import export_main_scene
from ..blueprints.export_blueprints import export_blueprints
from .export_materials import cleanup_materials, export_materials
from ..levels.bevy_scene_components import remove_scene_components, upsert_scene_components
@ -56,6 +56,7 @@ def auto_export(changes_per_scene, changed_export_parameters, settings):
#inject/ update light shadow information
for light in bpy.data.lights:
enabled = 'true' if light.use_shadow else 'false'
# TODO: directly set relevant components instead ?
light['BlenderLightShadows'] = f"(enabled: {enabled}, buffer_bias: {light.shadow_buffer_bias})"
# export
@ -69,7 +70,8 @@ def auto_export(changes_per_scene, changed_export_parameters, settings):
# since materials export adds components we need to call this before blueprints are exported
# export materials & inject materials components into relevant objects
if export_materials_library:
# FIXME: improve change detection, perhaps even add "material changes"
if export_materials_library and (changed_export_parameters or len(changes_per_scene.keys()) > 0 ):
export_materials(blueprints_data.blueprint_names, settings.library_scenes, settings)
# update the list of tracked exports

5
tools/blenvy/add_ons/auto_export/common/serialize_scene.py
View File

@ -235,7 +235,9 @@ def custom_properties_hash(obj):
custom_properties = {}
for property_name in obj.keys():
if property_name not in '_RNA_UI' and property_name != 'components_meta':
print("custom properties stuff for", obj, property_name)
custom_properties[property_name] = obj[property_name]
print("custom props for hashing", custom_properties, str(h1_hash(str(custom_properties))) )
return str(h1_hash(str(custom_properties)))
def camera_hash(obj):
@ -316,7 +318,7 @@ def modifiers_hash(object, settings):
def serialize_scene(settings):
cache = {"materials":{}}
print("serializing scene")
print("serializing scenes")
data = {}
@ -325,6 +327,7 @@ def serialize_scene(settings):
# TODO: only go through scenes actually in our list
for scene in bpy.data.scenes:
print("scene", scene.name)
# ignore temporary scenes
if scene.name.startswith(TEMPSCENE_PREFIX):
continue

58
tools/blenvy/add_ons/auto_export/levels/export_main_scenes.py → tools/blenvy/add_ons/auto_export/levels/export_levels.py
View File

@ -9,16 +9,8 @@ from ..constants import TEMPSCENE_PREFIX
from ..common.generate_temporary_scene_and_export import generate_temporary_scene_and_export, copy_hollowed_collection_into, clear_hollow_scene
from ..common.export_gltf import (generate_gltf_export_settings, export_gltf)
from .is_object_dynamic import is_object_dynamic, is_object_static
from ..utils import upsert_scene_assets
def assets_to_fake_ron(list_like):
result = []
for item in list_like:
result.append(f"(name: \"{item['name']}\", path: \"{item['path']}\")")
return f"(assets: {result})".replace("'", '')
return f"({result})".replace("'", '')
def export_main_scene(scene, settings, blueprints_data):
gltf_export_settings = generate_gltf_export_settings(settings)
@ -41,53 +33,7 @@ def export_main_scene(scene, settings, blueprints_data):
gltf_output_path = os.path.join(levels_path_full, scene.name)
inject_blueprints_list_into_main_scene(scene, blueprints_data, settings)
"""print("main scene", scene)
for asset in scene.user_assets:
print(" user asset", asset.name, asset.path)
for asset in scene.generated_assets:
print(" generated asset", asset)"""
"""for blueprint in blueprints_data.blueprints_per_scenes[scene.name]:
print("BLUEPRINT", blueprint)"""
blueprint_instances_in_scene = blueprints_data.blueprint_instances_per_main_scene.get(scene.name, {}).keys()
blueprints_in_scene = [blueprints_data.blueprints_per_name[blueprint_name] for blueprint_name in blueprint_instances_in_scene]
#yala = [blueprint.collection.user_assets for blueprint in blueprints_in_scene]
#print("dsfsdf", yala)
auto_assets = []
all_assets = []
export_gltf_extension = getattr(settings, "export_gltf_extension", ".glb")
blueprints_path = getattr(settings, "blueprints_path")
for blueprint in blueprints_in_scene:
if blueprint.local:
blueprint_exported_path = os.path.join(blueprints_path, f"{blueprint.name}{export_gltf_extension}")
else:
# get the injected path of the external blueprints
blueprint_exported_path = blueprint.collection['export_path'] if 'export_path' in blueprint.collection else None
# add their material path
materials_exported_path = blueprint.collection['materials_path'] if 'materials_path' in blueprint.collection else None
auto_assets.append({"name": blueprint.name+"_material", "path": materials_exported_path})#, "generated": True, "internal":blueprint.local, "parent": None})
if blueprint_exported_path is not None: # and not does_asset_exist(assets_list, blueprint_exported_path):
auto_assets.append({"name": blueprint.name, "path": blueprint_exported_path})#, "generated": True, "internal":blueprint.local, "parent": None})
# now also add the assets of the blueprints # TODO: wait no , these should not be a part of the (scene) local assets
for asset in blueprint.collection.user_assets:
#print("adding assets of blueprint", asset.name)
all_assets.append({"name": asset.name, "path": asset.path})
"""for asset in auto_assets:
print(" generated asset", asset.name, asset.path)"""
materials_path = getattr(settings, "materials_path")
current_project_name = Path(bpy.context.blend_data.filepath).stem
materials_library_name = f"{current_project_name}_materials"
materials_exported_path = os.path.join(materials_path, f"{materials_library_name}{export_gltf_extension}")
material_assets = [{"name": materials_library_name, "path": materials_exported_path}] # we also add the material library as an asset
print("material_assets", material_assets, "extension", export_gltf_extension)
scene["BlueprintAssets"] = assets_to_fake_ron(all_assets + [{"name": asset.name, "path": asset.path} for asset in scene.user_assets] + auto_assets + material_assets)
#scene["BlueprintAssets"] = assets_to_fake_ron([{'name':'foo', 'path':'bar'}])
upsert_scene_assets(scene, blueprints_data=blueprints_data, settings=settings)
if export_separate_dynamic_and_static_objects:
#print("SPLIT STATIC AND DYNAMIC")

84
tools/blenvy/add_ons/auto_export/utils.py Normal file
View File

@ -0,0 +1,84 @@
import bpy
import os
from pathlib import Path
from blenvy.assets.assets_scan import get_blueprint_asset_tree, get_main_scene_assets_tree2
def assets_to_fake_ron(list_like):
result = []
for item in list_like:
result.append(f"(name: \"{item['name']}\", path: \"{item['path']}\")")
return f"(assets: {result})".replace("'", '')
return f"({result})".replace("'", '')
# TODO : move to assets
def upsert_scene_assets(scene, blueprints_data, settings):
"""print("main scene", scene)
for asset in scene.user_assets:
print(" user asset", asset.name, asset.path)
for asset in scene.generated_assets:
print(" generated asset", asset)"""
"""for blueprint in blueprints_data.blueprints_per_scenes[scene.name]:
print("BLUEPRINT", blueprint)"""
blueprint_instances_in_scene = blueprints_data.blueprint_instances_per_main_scene.get(scene.name, {}).keys()
blueprints_in_scene = [blueprints_data.blueprints_per_name[blueprint_name] for blueprint_name in blueprint_instances_in_scene]
#yala = [blueprint.collection.user_assets for blueprint in blueprints_in_scene]
#print("dsfsdf", yala)
level_assets = []
all_assets = []
export_gltf_extension = getattr(settings, "export_gltf_extension", ".glb")
blueprints_path = getattr(settings, "blueprints_path")
for blueprint in blueprints_in_scene:
if blueprint.local:
blueprint_exported_path = os.path.join(blueprints_path, f"{blueprint.name}{export_gltf_extension}")
else:
# get the injected path of the external blueprints
blueprint_exported_path = blueprint.collection['export_path'] if 'export_path' in blueprint.collection else None
# add their material path
materials_exported_path = blueprint.collection['materials_path'] if 'materials_path' in blueprint.collection else None
level_assets.append({"name": blueprint.name+"_material", "path": materials_exported_path})#, "generated": True, "internal":blueprint.local, "parent": None})
if blueprint_exported_path is not None: # and not does_asset_exist(assets_list, blueprint_exported_path):
level_assets.append({"name": blueprint.name, "path": blueprint_exported_path})#, "generated": True, "internal":blueprint.local, "parent": None})
# now also add the assets of the blueprints # TODO: wait no , these should not be a part of the (scene) local assets
for asset in blueprint.collection.user_assets:
#print("adding assets of blueprint", asset.name)
all_assets.append({"name": asset.name, "path": asset.path})
"""for asset in level_assets:
print(" generated asset", asset.name, asset.path)"""
materials_path = getattr(settings, "materials_path")
current_project_name = Path(bpy.context.blend_data.filepath).stem
materials_library_name = f"{current_project_name}_materials"
materials_exported_path = os.path.join(materials_path, f"{materials_library_name}{export_gltf_extension}")
material_assets = [{"name": materials_library_name, "path": materials_exported_path}] # we also add the material library as an asset
print("material_assets", material_assets, "extension", export_gltf_extension)
all_assets_raw = get_main_scene_assets_tree2(main_scene=scene, blueprints_data=blueprints_data, settings=settings)
local_assets = [{"name": asset["name"], "path": asset["path"]} for asset in all_assets_raw if asset['parent'] is None and asset["path"] != "" ]
all_assets = [{"name": asset["name"], "path": asset["path"]} for asset in all_assets_raw if asset["path"] != "" ]
print("all_assets_raw", all_assets_raw)
print("all_assets", all_assets)
print("local assets", local_assets + material_assets)
scene["BlueprintAssets"] = assets_to_fake_ron(local_assets + material_assets)
#scene["BlueprintAssets"] = assets_to_fake_ron(all_assets + [{"name": asset.name, "path": asset.path} for asset in scene.user_assets] + level_assets + material_assets)
#scene["BlueprintAssets"] = assets_to_fake_ron([{'name':'foo', 'path':'bar'}])
def upsert_blueprint_assets(blueprint, blueprints_data, settings):
all_assets_raw = get_blueprint_asset_tree(blueprint=blueprint, blueprints_data=blueprints_data, settings=settings)
all_assets = []
auto_assets = []
local_assets = [{"name": asset["name"], "path": asset["path"]} for asset in all_assets_raw if asset['parent'] is None and asset["path"] != "" ]
print("all_assets_raw", all_assets_raw)
print("local assets", local_assets)
blueprint.collection["BlueprintAssets"] = assets_to_fake_ron(local_assets)

24
tools/blenvy/assets/assets_scan.py
View File

@ -121,6 +121,30 @@ def get_main_scene_assets_tree(main_scene, blueprints_data, settings):
added_asset.path = asset["path"]
return assets_list
# same as the above, withouth the clutter below : TODO: unify
def get_main_scene_assets_tree2(main_scene, blueprints_data, settings):
blueprints_path = getattr(settings, "blueprints_path")
export_gltf_extension = getattr(settings, "export_gltf_extension", ".glb")
blueprint_instance_names_for_scene = blueprints_data.blueprint_instances_per_main_scene.get(main_scene.name, None)
assets_list = get_user_assets_as_list(main_scene)
if blueprint_instance_names_for_scene:
for blueprint_name in blueprint_instance_names_for_scene:
blueprint = blueprints_data.blueprints_per_name.get(blueprint_name, None)
if blueprint is not None:
blueprint_exported_path = None
if blueprint.local:
blueprint_exported_path = os.path.join(blueprints_path, f"{blueprint.name}{export_gltf_extension}")
else:
# get the injected path of the external blueprints
blueprint_exported_path = blueprint.collection['export_path'] if 'export_path' in blueprint.collection else None
if blueprint_exported_path is not None and not does_asset_exist(assets_list, blueprint_exported_path):
assets_list.append({"name": blueprint.name, "path": blueprint_exported_path, "type": "MODEL", "generated": True, "internal":blueprint.local, "parent": None})
assets_list += get_blueprint_assets_tree(blueprint, blueprints_data, parent=blueprint.name, settings=settings)
return assets_list
def get_blueprint_asset_tree(blueprint, blueprints_data, settings):
blueprints_path = getattr(settings, "blueprints_path")
export_gltf_extension = getattr(settings, "export_gltf_extension", ".glb")