public static Vector2[] GetElectrodePositions(this ElectrodePattern that, ElectrodeLayout layout, float fov = 120) { switch (that) { case ElectrodePattern.POLYRETINA: return(Polyretina(layout, fov)); case ElectrodePattern.ArgusII: return(ArgusII()); default: return(null); } }
public bool IsPhospheneTextureReady(HeadsetModel headset, ElectrodePattern pattern) { var ready = true; foreach (var layout in Enumerate <ElectrodeLayout>()) { ready &= IsPhospheneTextureReady(headset, pattern, layout); } return(ready); }
void OnEnable() { _dataType = DataType.Phosphene; _headset = HeadsetModel.VivePro; _pattern = ElectrodePattern.POLYRETINA; _layout = ElectrodeLayout._80x150; _path = "Assets/"; _gpuAccel = true; cpuStarted = false; }
/* * Public methods */ /// <summary> /// Creates a data texture. A threadGroup is returned to keep track of creation progress; Null if GPU accelerated. /// </summary> public static ThreadGroup Create(DataType type, HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout, bool gpuAccel, string path) { if (gpuAccel) { StartGPU(type, headset, pattern, layout, path); return(null); } else { return(StartCPU(type, headset, pattern, layout, path)); } }
void OnGUI() { UnityGUI.Header("Coordinates"); headset = UnityGUI.Enum("Headset", headset); UnityGUI.Space(); UnityGUI.BeginHorizontal(); UnityGUI.Label(" ", "X", UnityGUI.BoldLabel); UnityGUI.Label("Y", UnityGUI.BoldLabel); UnityGUI.EndHorizontal(); UnityGUI.BeginHorizontal(); UnityGUI.Label("Resolution", headset.GetWidth().ToString()); UnityGUI.Label(headset.GetHeight().ToString()); UnityGUI.EndHorizontal(); UnityGUI.BeginHorizontal(); UnityGUI.Label("Field of View", ((int)headset.GetFieldOfView(Axis.Horizontal)).ToString()); UnityGUI.Label(((int)headset.GetFieldOfView(Axis.Vertical)).ToString()); UnityGUI.EndHorizontal(); UnityGUI.BeginHorizontal(); UnityGUI.Label("Diameter (um)", ((int)headset.GetRetinalDiameter(Axis.Horizontal)).ToString()); UnityGUI.Label(((int)headset.GetRetinalDiameter(Axis.Vertical)).ToString()); UnityGUI.EndHorizontal(); UnityGUI.Space(); Draw("Pixel", ref pixel); Draw("Visual Angle", ref angle); Draw("Retina (um)", ref retina); Draw("Polar", ref polar); UnityGUI.Separator(); UnityGUI.Header("Electrode Count"); UnityGUI.BeginChangeCheck(); pattern = UnityGUI.Enum("Pattern", pattern); layout = UnityGUI.Enum("Layout", layout); fieldOfView = UnityGUI.Float("Field of View", fieldOfView); var changed = UnityGUI.EndChangeCheck(); if (changed) { numElectrodes = pattern.GetElectrodePositions(layout, fieldOfView).Length; } UnityGUI.Space(); UnityGUI.Label("Electode Count", numElectrodes.ToString()); }
/* * Private methods */ private int GetPhospheneTextureIndex(HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout) { var numPatterns = Count <ElectrodePattern>(); var numLayouts = Count <ElectrodeLayout>(); var headsetIndex = headset.GetIndex(); var patternIndex = pattern.GetIndex(); var layoutIndex = layout.GetIndex(); var headsetOffset = headsetIndex * (numPatterns * numLayouts + 1); var patternOffset = patternIndex * numLayouts; return(headsetOffset + patternOffset + layoutIndex); }
private static ThreadGroup StartCPU(DataType type, HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout, string path) { // declare colour matrix and electrodes (if needed) var matrix = new Color[headset.GetWidth(), headset.GetHeight()]; var electrodes = type == DataType.Phosphene ? pattern.GetElectrodePositions(layout) : default; // setup threadGroup var threadGroup = new ThreadGroup(); threadGroup.OnAllThreadsFinished += () => { EditorCallback.InvokeOnMainThread(() => { var asset = headset.CreateTexture(); asset.SetPixels(matrix.Flatten(false, true, true)); asset.Apply(); SaveAsset(asset, path); }); }; // generate the texture asynchronously threadGroup.ProcessArray(headset.GetWidth(), (i) => { var n = headset.GetHeight(); for (int j = 0; j < n; j++) { switch (type) { case DataType.Phosphene: matrix[i, j] = PhospheneMatrix.CalculatePoint(i, j, headset, electrodes).ToColour(); matrix[i, j] = AddRandomSeeds(matrix[i, j]); break; case DataType.Axon: matrix[i, j] = AxonMatrix.CalculatePoint(i, j, headset).Colour; break; } } }); // provide caller with threadGroup for completion statistics return(threadGroup); }
void OnGUI() { UnityGUI.Space(); _dataType = UnityGUI.Enum("Data Type", _dataType); UnityGUI.Space(); UnityGUI.Label("Parameters", UnityGUI.BoldLabel); _headset = UnityGUI.Enum("Headset Model", _headset); if (_dataType == DataType.Phosphene) { _pattern = UnityGUI.Enum("Electrode Pattern", _pattern); _layout = UnityGUI.Enum("Electrode Layout", _layout); } UnityGUI.Space(); _gpuAccel = UnityGUI.ToggleLeft("GPU Accelerated", _gpuAccel); if (UnityGUI.Button(!cpuStarted ? "Start" : "Stop", new GUIOptions { maxWidth = 50 })) { if (!cpuStarted) { Start(); } else { StopCPU(); } } if (threadGroup != null) { UnityGUI.Enabled = cpuStarted || !_gpuAccel; UnityGUI.Space(); UnityGUI.Label("CPU Threads", UnityGUI.BoldLabel); for (int i = 0; i < threadGroup.NumThreads; i++) { UnityGUI.Label("Thread " + i.ToString(), (threadGroup.Progress[i] * 100).ToString("N0") + "%"); } } }
/* * Protected methods */ protected void Initialise(string phosShader, string tailShader) { // load shaders phos = new Material(Shader.Find(phosShader)); tail = new Material(Shader.Find(tailShader)); if (phos == null || tail == null) { Debug.LogError($"{name} does not have a material."); return; } // upload electrode/axon textures to the GPU phos.SetTexture(SP.electrodeTexture, epiretinalData.GetPhospheneTexture(headset, pattern, layout)); tail.SetTexture(SP.axonTexture, epiretinalData.GetAxonTexture(headset)); // create texture for the fading data fadeRT = new DoubleBufferedRenderTexture(headset.GetWidth(), headset.GetHeight()); fadeRT.Initialise(new Color(1, 0, 0, 0)); // overrides if (overrideCameraFOV) { Prosthesis.Instance.Camera.fieldOfView = headset.GetFieldOfView(Axis.Vertical); } if (overrideRefreshRate) { Application.targetFrameRate = headset.GetRefreshRate(); } // cache texture-related variables lastHeadset = headset; lastPattern = pattern; lastLayout = layout; // initialise eye gaze tracking EyeGaze.Initialise(eyeGazeSource, headset); // initialise camera target eye UpdateCameraTargetEye(); }
/* * Private methods */ private static void StartGPU(DataType type, HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout, string path) { var shader = LoadAsset <ComputeShader>($"{type}"); var kernel = shader.FindKernel("CSMain"); var electrodeBuffer = default(ComputeBuffer); if (type == DataType.Phosphene) { var electrodes = pattern.GetElectrodePositions(layout); electrodeBuffer = new ComputeBuffer(electrodes.Length, sizeof(float) * 2); electrodeBuffer.SetData(electrodes); shader.SetBuffer(kernel, "_electrodes", electrodeBuffer); } var texture = headset.CreateRenderTexture(); shader.SetTexture(kernel, "_result", texture); shader.SetVector("_headset_diameter", headset.GetRetinalDiameter()); shader.SetVector("_headset_resolution", headset.GetResolution()); shader.Dispatch(kernel, headset.GetWidth() / 8, headset.GetHeight() / 8, 1); electrodeBuffer?.Dispose(); Texture2D asset = texture.ToTexture2D(TextureFormat.RGBAFloat, true); texture.Release(); asset.anisoLevel = 0; asset.filterMode = FilterMode.Point; if (type == DataType.Phosphene) { AddRandomSeeds(asset); } SaveAsset(asset, path); }
public bool IsPhospheneTextureReady(HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout) { return(GetPhospheneTexture(headset, pattern, layout) != null); }
public void SetPhospheneTexture(HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout, Texture2D asset) { data[GetPhospheneTextureIndex(headset, pattern, layout)] = asset; }
/* * Public methods */ public Texture2D GetPhospheneTexture(HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout) { return(data[GetPhospheneTextureIndex(headset, pattern, layout)]); }
private void UpdatePerChangeData() { // // textures are only updated when necessary for efficiencies sake // // axon texture if (headset != lastHeadset) { tail.SetTexture(SP.axonTexture, epiretinalData.GetAxonTexture(headset)); if (overrideCameraFOV) { Prosthesis.Instance.Camera.fieldOfView = headset.GetFieldOfView(Axis.Vertical); } if (overrideRefreshRate) { Application.targetFrameRate = headset.GetRefreshRate(); } lastHeadset = headset; } // phosphene texture if (headset != lastHeadset || pattern != lastPattern || layout != lastLayout) { phos.SetTexture(SP.electrodeTexture, epiretinalData.GetPhospheneTexture(headset, pattern, layout)); lastHeadset = headset; lastPattern = pattern; lastLayout = layout; } // // everything else is updated every frame // there is already per-frame data that needs to be uploaded to graphics card anyway (e.g., eye gaze, pulse), // so adding a few more floats probably isn't a big performance hit (probably, definitely not tested) // // headset diameter var headsetDiameter = headset.GetRetinalDiameter(); phos.SetVector(SP.headsetDiameter, headsetDiameter); tail.SetVector(SP.headsetDiameter, headsetDiameter); // electrode radius phos.SetFloat(SP.electrodeRadius, layout.GetRadius(LayoutUsage.Anatomical)); // implant radius var implantRadius = CoordinateSystem.FovToRetinalRadius(fieldOfView); phos.SetFloat(SP.polyretinaRadius, implantRadius); tail.SetFloat(SP.polyretinaRadius, implantRadius); // brightness phos.SetFloat(SP.brightness, brightness); // luminance levels phos.SetInt(SP.luminanceLevels, luminanceLevels); // luminance boost var range = 1 - (1f / luminanceLevels); phos.SetFloat(SP.luminanceBoost, luminanceBoost * range); // variance phos.SetFloat(SP.sizeVariance, sizeVariance); phos.SetFloat(SP.intensityVariance, intensityVariance); phos.SetFloat(SP.brokenChance, brokenChance); // decay constant tail.SetFloat(SP.decayConst, tailLength); // update decay/recovery fading parameters UpdateDecayParameters(decayT1, decayT2, decayThreshold); UpdateRecoveryParameters(recoveryDelay, recoveryTime, recoveryExponent); // keywords UpdateKeyword("USE_FADING", useFading); UpdateKeyword("RT_TARGET", Prosthesis.Instance.Camera.targetTexture != null); UpdateKeyword("OUTLINE", outlineDevice); UpdateTailQuality(); UpdateTargetEye(); }
/* * Public methods */ public static Vector2[,] CalculateMatrix(HeadsetModel headset, ElectrodePattern pattern, ElectrodeLayout layout) { return(CalculateMatrix(headset, pattern.GetElectrodePositions(layout))); }