Can you tell us more about your 2D to 3D conversion app ?

a little background 1st..
For many years I've wanted a software version of what the LG 3D TV's have in their Firmware.
Real time 2D to 3D conversion.

have a look Here:
https://iwantaholodeck.com/stream-to-3d-configuration/
and Here
https://iwantaholodeck.gumroad.com/l/wnclr

I thought I found the golden Ticket...
I bought in...
I was totally Disappointed, It's not 3D, it's a 2D screen Floating

So I decided to Dive in and see what I could come up with..
After a few hours I had a pretty good working Live 2D to 3D app.

I've tried to package it into a single Downloadable Executable file.
But so far everything I have tried Breaks it..

I have it running on my system using my GPU's CUDA cores for all the Heavy processing.
but my GPU is aged, an RTX 3060, and I'm not running the Latest Drivers.

Every time I bundle it up, the resulting executable Defaults to CPU only
and that runs at about 2 Seconds per Frame, or 30 Frames a minute.

With my GPU, I get a pretty Steady 24fps Live 2D to 3D conversion.

I do a Find edges with High Contrast.
then I normalize the Edges to White

I do a Gradient Flood fill on the Inside shape of the Edges.
Depending where on the Screen the Shape is Found, the Gradient shift is Darkened to account for Depth.
Then I make an Inverted Gray Scale of the Orig Image, and do a weighted Blend
of the Gradient, and the Gray Scale.

the Find edges get inverted and Feathered as an Edge Fall off to give the shapes edge Definition

finally this pseudo Depth Map is applied as a Pixel Shift where White is high and Black is Low.

I frame buffer up to 4 frames
and pop them off the stack to use as the Alternate EYE image
So that each eye has a slightly different Shifted Image.

It's all just a lot of weighted Math.
combined with a bit of trial and error..

I've tried to Video while I'm doing the On the Fly Conversion, but my Systems just Doesn't have the Balls to Do both at the Same time.

PS
Using what I learned, I did my Best to incorporate some of the in my Latest Scene
There is am extra Folder with Two additional Scenes
These apply my 2Dto3D shader
the results are just ok, not great, not terrible

The Shader provides for some debug Views to look at the Edges, and the Gradient
The Problem with a shader is that It's a ONE pass process.
I can not find edges, and Store them to later reuse with a Flood Fill.
Edge Detection is along a Single Horizontal Scan Line
and then I can Fill in between these Edge Pixels, with a Gradient Line of Pixels.
Repeat for the Next Row
then Stack all the rows together and you get a Single Pass
But the Gradient isn't the Same As if you Had the Shape Outline, and did a Flood fill to get a Smooth coherent Gradient.

What you get, is a Jagged gradient, which with a blur can be sort of blended.

Ok I attempted to record a video of doing 2Dto3D on the Fly conversion.
https://wyldanimal.com/2Dto3D/2dto3D-demo-002.mp4

I also
made 54 different test videos

You Have to Download them to watch them...
is-001_3D.mp4 to is-054_3D.mp4
just change the 3 Digit number in the Link
https://wyldanimal.com/2Dto3D/is-001_3D.mp4

https://wyldanimal.com/2Dto3D/is-001_3D.mp4

PS, I'm hoping that OWL3D will soon have a live version
Their AI keeps getting Faster...

@WA

That's very intersting - thanks for sharing all of that. I tried to get python to work with CUDA for a different project but I couldn't get it to compile - tried many times and gave up.

If I understand what you've done correctly, you've taken a different route to me to get a 3D effect.

A few questions....

Are you taking the whole feed from a non SBS screen and converting it to SBS with the depth processing ? i.e you're not starting from a SBS view of iStripper ?

Doe the .py program analyse the whole frame and say.... looking at the gradient and edges, I will move this pixel n steps ? If it does and you get gaps, how do you fill them ?

I'm assuming this has no concept of depth from shape, so for example the side walls of a room aren't given any extra depth other than that derived from their gradient and edge effects ?

Are you taking the whole feed from a non SBS screen and converting it to SBS with the depth processing ? i.e you're not starting from a SBS view of iStripper ?

Correct, I'm taking the Entire 2D view, 2D background and 2D model, exactly what is seen in the OBS view a screen 2.
I capture a frame from Screen 2, apply my 3D transition, Pixel Shift the Right eye
Display the Capture as the Left Eye and the Pixel shifted image as the right eye.
I also do Frame buffering, so that the Left eye is one Frame or multiple Frames ahead of the Right Eye.
A Zero Frame Buffer Means, the 3D effect is Purely from the Applied Gradient.

A One frame Buffer, means that the Right eye, has Pixels from the Previous frame included.

"Doe the .py program analyse the whole frame and say.... looking at the gradient and edges, I will move this pixel n steps ? If it does and you get gaps, how do you fill them ?"

yes the entire frame is analyzed.
A blur to the overall gradient is applied, the edges are softened and applied to give edge definition.
It's N steps times the Shift Amount. The Gray value of the Gradient controls N, and the Shift Amount is the Multiplier.
Shift can be Positive or Negative, and the Gradient can also Be Inverted.

I'm assuming this has no concept of depth from shape, so for example the side walls of a room aren't given any extra depth other than that derived from their gradient and edge effects ?

I start at the Bottom, and Scan upwards, each Edge is assigned a gray Gradient based on how far from the bottom it is.

Gradients have a color Range, say 1 to 10, and then the Next Level Gradient has a Gap say of 30
so then the very Next gray level would start at 40 10 + gap of 30 = 40 to 50
So these two levels would be
Gradient 1 is 1 to 10
Gap of 30
Gradient 2 is 40 to 50
Gap of 30
Gradient 3 is 80 to 90
repeat
After all the Gradients are built, an Over all blur is applied, and then the soft Edge detections are blended back in
This becomes the Pseudo Depth Map
Depending on the Mode...
a Gray Scale of the Captured Frame is also Blended in or not..

Most All of these can be tweaked on the fly.

In addition to the Live 2Dto3D
I included a File Conversion
where the Video Frames are read from a video File, conversion applied, and then displayed, And / Or Saved to an Ouput File.

@WA

Thanks for sharing all of that. There's lots going on here but I think it's only you and me that seem to be interested in this mind blowing tech. lol.

Here's a new demo clip of where I've got to with my own 3D shader. The last one of these I posted was downloaded 63 times without a single comment from anyone - so I kind of gave up on the forum at that point and decided that I would remain the only person in the world viewing iStripper in this amazing way.

Just to be clear this is a realtime shader, no post processing.

Anyone interested (including @Totem) pls respond.

https://bit.ly/4pjP9ki

I went back to look, and I saw that I did miss one of your Scenes..
(8K)(SBS) Gallery

Is you current Demo video Made using that Scene or do you have an updated Scene you are using?

I had some trouble running it on my system, but saw enough to Like it...

I couldn't play the Scene Plus view it with Virtual Desktop at the same time.
So I just screen captured it running, and then Played the video in the Headset..

It's a while since I produced this demo but I haven't uploaded the .fsh so if you are looking at one of my .scn files then that's OLD OLD and just has a similar name.

What is in the demo .mpg is the 3D enhancement to the models not the backgrounds. That's the key thing here.... models with texture. Hooters that hoot, clothes that have wrinkles oh and of course wrinkly bits with wrinkles.

I use virtual desktop's video player to view this with no issues.

I was so impressed by Dolly Orchids - Neon ***** outfit that I decided to make some 3D backgrounds for her. Here's a very quick sample of her demo clips for this card in 3D SBS - best viewed with Virtual Desktop video player

https://bit.ly/3IhxkBV

I did about 20 different backgrounds with a "Neon ***** " theme. If anyone wants them in either 2D or 3D ... just ask.

https://bit.ly/3IhxkBV

I did about 20 different backgrounds with a "Neon ***** " theme. If anyone wants them in either 2D or 3D ... just ask.

Just watched it.
Excellent

If you don't mind Sharing, both the 2D and the 3D versions would be nice.

PS: I've put in a Feature request on the Discord Server.
some support from other 3D enthusiasts would be Great!

https://discord.com/channels/709339443777110026/1418620490125152366/1418620490125152366

I was so impressed by Dolly Orchids - Neon ***** outfit that I decided to make some 3D backgrounds for her. Here's a very quick sample of her demo clips for this card in 3D SBS - best viewed with Virtual Desktop video playerhttps://bit.ly/3IhxkBVI did about 20 different backgrounds with a "Neon ***** " theme. If anyone wants them in either 2D or 3D ... just ask.

@Calgon

Do you have the 3D background version of this without the SBS video setup? When I use SBS video setup it starts to make me a bit dizzy.

@LampShade

Do you have the 3D background version of this without the SBS video setup?

Did you mean a 2D version or a 3D version with some other type of screen arranagment ?

For the SBS version, assuming you are using virtual destop, I adjust the screen,,, biger or smaller...near or far... until I find somewhere comfortable.

@Calgon

2D version. The camera is set as 3D in the .scn file.

@LampShade

The camera is set as 3D in the .scn file.

A 3D camera doesn't make the scene look 3D it is just a way of "positioning" the elements of the scene. A 2D camera scene only has x and y coordinates but you can adjust size, a 3D camera setup has x,y,z coordinates so you can make something bigger by moving it closer on the z axis. I use a combination of these in that .scn 2x 3D cameras to build the view from each eye's perspective and 1x 2D camera to stick the components together.

A quick hack to get back to 2D... If you find the 3rd camera in the scene file the bit that looks like this:

// Main camera to combine the framebuffers for stereoscopic output
camera {
type: 2D
id: MainCamera
// size: 3840, 2160 // Full 4K screen width and height
// size: 3664, 1920 // Full 8K screen width and height !! Virtual Desktop will downscale it for us !

// size: 4580, 2400 // oc plus
size: 7680, 4320 // oc plus







// Render the left framebuffer on the left side of the screen with inversion correction
sprite {
//pos: 960, 1080, 0 // Position the Left Eye framebuffer on the left
pos: 1145, 1200 // Position the Left Eye framebuffer on the left
pos: 1920, 2160
//size: 1920, 2160 // Full height for 4K
//size: 3840, 4320 // Full height for 4K
size: 2290, 2400
size: 3840, 4320
source: LHS_Buffer
scale: 1.0, -1.0 // Flip vertically to counteract inversion

}

// Render the right framebuffer on the right side of the screen with inversion correction
sprite {
//pos: 960, 1080, 0 // Position the Left Eye framebuffer on the left
pos: 3435, 1200 // Position the Left Eye framebuffer on the left
pos: 5760, 2160 // Position the Left Eye framebuffer on the left
//size: 1920, 2160 // Full height for 4K
//size: 3840, 4320 // Full height for 4K
size: 2290, 2400
size: 3840, 4320
source: RHS_Buffer
scale: 1.0, -1.0 // Flip vertically to counteract inversion
}






}

and replace it with this:

camera {
type: 2D
id: MainCamera
size: 7680, 4320 // oc plus
// Render the left framebuffer on the left side of the screen with inversion correction
sprite {
pos: 3840, 2160
size: 7680, 4320
source: LHS_Buffer
scale: 1.0, -1.0 // Flip vertically to counteract inversion
}
}

...that will give you the left eye view across the whole screen.

@Calgon

Thanks, that worked. :)

Quick question, in the .scn file for bounce, what is node and how does it work? Is node an extra animation location?

@LampShade

A node is an Unnamed Group
It combines many different elements in to One Group that can be acted on all Together

You Can Animate the Node
You can Apply a Shader to the Node
You can Position the Node
You can Scale The Node
You Can rotate the Node

All elements in side the Node act as a single Element.

In the Bounce Scene, an Animation is Applied to the Node
and then Everything in side of that Node is Animated.

an Outlined example:
Node{ <-- opening bracket of a Node
Animate: blah Blah Blah.....

Sprite{
}
Sprite{
}
Sprite{
}

} <-- closing Barcket of the Node

All Three Sprites inside of the Node will be Animated together.

node can be used in the Camera View
or they can be used in a FrameBuffer


Hope that Helps

Using the same example:

an Outlined example:
Node{ <-- opening bracket of a Node

Sprite{
}
Sprite{
}
Sprite{
}

Animate: blah Blah Blah.....
} <-- closing Barcket of the Node

All Three Sprites inside of the Node will be Animated together.

The animate function can be at the Top of the Node, or Bottom of the Node
It has the same Affect on the Element inside the Node.