Sensation and Perception Part 3

• Prepares
• Selects
• Organizes
• Integrates Sensations

• Novelty (Relevant info)
• Movement
• Intensity
• Contrast

• adjustment to important info, specific stimuli
• adjustment of eyes, ears, etc.

isolating awareness to one thing

paying attention to 2+ objects at once

• focusing on certain subsets of info, filtering out others 
• cocktail party phenomenon
• selective looking - visual search

• Central task- focus on stimuli that answers question, very accurate
• Peripheral task - stimuli that answers question not in focus, very accurate
• Dual task - stimulus in peripheral, but also secondary stimuli (disk or face), accurate for face
• Face has meaning, does not need focused attention to be aware of it, disk does

• two messages going in each ear, attention paid to one, what changes are noticed in the other?
• 1: noticed gender change and lack of message, not content or reversing
• 2: message switch ears
• 3: notices name in the ignored message

• how does attention affect awareness and understanding of stimuli?
• If focused on stimuli to answer question, intro. of secondary stimuli not noticed (blind) 60%-80%

• Hard to notice changes in information-heavy scene
• If focused on one thing (map and directions), wont notice big change elsewhere (tourist asking for direction)

Background awareness, brief exposure (Rapid Serial Visual Presentation (RSVP)) gives a gist, but specifics nor the scene are dedicated to memory

• Selective attention leading to selective awareness (and thus blindness), where specifics of scene are dedicated to memory, others are filtered out
• emotional or relevant info often gets attention

• If cue is given in direction of future stimulus, response time (RT) to pay attention to stimulus decreased
• Neutral cue has medium RT
• Cue opposite of future stimulus increases RT

• 1: Effective (high firing rate) and ineffective (low firing rate) stimuli presented, attention changed from effective to ineffective
• Only the stimuli given attention affected neuron firing rate
• 2: Attention shift into and out of receptor field (RF) following cue
• Neurons firing rate changed with cue, before stimulus changed
• Neuron firing rate (and thus attention) doesn't need eye movement to show change

Selective looking, one specific feature, easy, fast, number of items doesn't matter, use parallel search (equal attention) over whole field

Selective features, multiple features, difficult, RT increases WITH number of items, requires serial search over field

• Once features are seen, how to match (bind) features with objects in field
• Distributed representation: integrate distributed brain signals responding to different distributed features

• Fixes binding problem, as when we only pay attention to one single object, the features of the object easily integrated back to that object
• (conjunction search)

• If features of different objects are in single RF, the features compete for representation
• Competition increases as complexity of representation increases (up visual pathway)
• Selectively pay attention to one object to bias representation to this object

• Top-Down- info. suggests where to pay attention, voluntary control of attention, slow
• Bottom-Up- stimulus demands attention (reflex) involuntary control, fast response

Damage leads to unilateral (left) field neglect, like V4, neurons respond to cue for attention shift, whether or not eye movement ocurrs

• Frontal cortex
• Controls eye movement as well as attention shifting without eye movement

• fixation- no attention shift, baseline firing rate
• saccade- attention shift with eye movement, firing increase before eye movements
• peripheral- attention shift in peripheral (no eye movement), firing rate still increases before shift (with cue)

• Parietal lobe damage, cant focus attention to connect features (necessary for conjunction search)
• Parietal- Petyr Balint- Balish - connects the dots

How to relate awareness with brain activity - correlate certain neuron features with shift in awareness (perceptual bistability) or lack of awareness (blindsight)

• Binocular Rivalry: Present two different images to each eye, shift awareness between images as they alternate 
• (house-face images, house-parrahippocampal activity, face- fusiform gyrus activity)
• Neural Correlate of Conciousness

• Recognize, discriminate, and react to visual stimulus (pupillary reflex, eye movement) without awareness of that stimuli (caused by V1 damage)
• Reaction to stimulus seen in amygdala (emotional stimulus) and in superior colliculus and other visual areas with corollary discharge theory
• Neural Correlate of Conciousness

Constantly switching attention and thus tasks, requiring time in between each shift

• visual encoding
• grouping encoding
• object encoding
• search (increase search time)

• distract or divert attention
• prevent detection
• blend with environment
• Lateral Inhibition and Edge Detection prevention

• Conceal oneself
• Mimic another subject
• Deceive searcher
• Blend with background
• Hiding is NOT an example of camo.

• become as similar as possible to something else, prevent recognition or detection (mimic background)
• behavior
• looks
• smell
• sound
• location

• prevention of detection while in plain sight (NOT hiding)
• countershading
• disruptive patterns and colors
• matching background

• random but strong colors, preventing detection of continuous structure (pattern-black and white stripes on zebra)
• lose structure continuity by preventing edge detection

looking like (mimicing) something else to prevent recognition, not detection

• Matching background and light differences, dark top side matches light bottom side after light hits top and casts shadow on bottom side
• 1. delete own shadow
• 2. match one or multiple backgrounds (two backgrounds, two directions)
• 3. change shading to change 3D detection (UV light)
• 4. obliterate shadow to prevent 3D detection

• Concealing-Coloration
• Negative shading aka Upside-down shading aka countershading
• Dazzle Paint (confuse motion -speed and direction, confuse shape - fake ship front)

surfaces reflect light at certain spectral power distributions and intensities, creating intensity borders

• related to material properties 
• Illumination edge - shadow with short WL dark light, UV detectable
• Internal marking - texture pattern of reflected light

• Mie and Rayleigh Scattering affecting light spectrum visible
• Mie scattering- atmosphere near sun appears sun color
• color changes quickly - border
• color and intensity quickly - border unless shadow (illumination edge)
• change over time - assign borders by light change, detect motion and assign common fate

• Alter/disrupt small edges with color/pattern
• Delete or add extraneous edge information

• Edge detector neurons like strong difference separation
• Blur edges with graded pigmentation, creates perpendicular or adjacent strong "fake" edges to disguise real one called ILLUSORY CONTOURS

• Optic Flow Mimicry
• Motion Signal Minimization
• Motion Disruption

prevent detection of motion by minimizing either motion itself or the signal it creates

change form (illusory contours) or motion cues to alter motion perception

• Blend with moving background as searcher moves through environment
• Real point and infinity point strategies
• Dragonfly and Bat do both strategies

• Prevent background discontinuity to remove 3D shape
• Produce 2D pattern on surface to remove 3D shape

• Efficient searching requires serial search over multiple features/objects
• The more varied the environment and the more varied the camouflaged object is, the longer the search

• Modify call (frequency, pattern, structure, etc)
• Prevent detection by minimization
• Mimic sound of other subject
• Prevent localization (throw voice)

Insects mimic chemicals or limit chemicals detected (caterpillar mimics twig scent for ants)

• signals used for mating and coupling
• signals used to detect predator or prey

• Fish can detect change in electric fields
• Passive detection - detect change in electric fields by movement of other objects
• Active detection - send out electric signal like sonar
• Send electric signal to match lightning to prevent detection

• Sound detection in horizontal plane by two processes
• Interaural Level Difference
• Interaural Time Difference

• Perception - Experience of hearing
• Sensation - pressure changes of medium producing detectable waves in ear

• Diaphragm condenses or compresses air, pushing it out (increasing pressure)
• Diaphragm rarefacts in air, pulling it back in (decreasing pressure)

• Amplitude (decibel, dB)
• Frequency (proportional to wavelength) (hertz -Hz)
• Waveform (complexity)

• Loudness
• Pitch
• Timbre

periodic sound wave called a sine wave

• decibels, loudness doubles every 10 dB, pressure max at top of wave
• 85 dB max safe level (8 hr. exposure)
• max at 4000 Hz

• Hertz, pitch changes one octave every time frequency doubles
• Range- 20 - 20000 Hz
• Best at 2000 Hz

Starts with fundamental frequency (first harmonic), all other frequencies in complex sound (harmonics) multiple of first harmonic

Adding together harmonics (frequencies) to complexify sounds

Display of each harmonic (frequency) in complex sound

Beginning of sound that builds up and the end of the sound that dies off

• complexity of sound, determines all other features besides pitch and loudness and duration, allowing for sound distinction between sources
• Removing first harmonic (fundamental frequency) creates different timbre but retains perceived pitch

Displays complex sound as frequency components as their own each sine wave

Absorbs all sound, preventing any sound reflection, lowest decibels (99.9% sound absorbed)

reflects all sound possible for as long as possible, measures sound recorders and sound producers

• Compares how easy to detect sound (absolute threshold curve) at each frequency
• best (lowest threshold) at 2000-4000 Hz

• Compares how loud we percieve a sound at different frequencies
• compare to 1000 Hz standard tone
• Equal loudness of frequencies at 80 dB
• Low and High frequencies softer at 40 dB

Energy of sound decreases with distance squared

• Vestibular Canal connected to oval window, receiving sound first before sending to Tympanic Canal
• Vestibular - Vest - Best - First

Tympanic canal connected to round window, receives sound from vestibular canal

• Between vestibular canal and tympanic canal
• containing organ of corti, basilar membrane, and tectorial membrane

Separates vestibular canal and choclear duct

separates choclear duct and tympanic membrane

fluid that fills scala vestibular and scala tympani in choclea

space at tip of outstretched basilar membrane in choclea

fluid filling organ of corti, assisting with transduction of sound from hair cells to auditory nerve

• Activated by bending of stereocilia on outer hair cells
• allowing for amplification and sharpening of basilar membrane and thus sound transduction

• How- Firing rate corresponds to frequency
• Which - selective neurons only fire at certain frequencies, based on selective hair cells on basilar membrane

• Place (code) theory - certain hair cells respond based on location on basilar membrane and which frequency vibrates basilar memrane 
• doesn't work for low Hz, doesn't move membrane
• Frequency (temporal code) theory - neuron fire rates correspond to frequency

• Depiction indicating power of movement created by wave as wave moves across basilar membrane
• detected frequency is location of where wave is at peak power

mapping frequency (high to low) on basilar membrane (base to apex) across choclea

• representation of detected threshold of various frequencies
• presented by pure tones with white noise (masked threshold)
• showing the detection sensitivity along basilar membrane
• For higher frequencies, cuves become narrower, showing greater sensitivity
• outer hair cells have slight curve, indicating amplification effect

• temporal place code theory explanation,
• neurons respond with fire rate corresponding to frequency
• "phase-lock" with peak of sine wave to fire
• Groups of fibers needed, fire in bursts with pauses

• range of amplitudes audible and whose frequencies we can discriminate between
• limited by maximum firing rate of neurons (grouped together to increase max level)

• range of amplitudes a neuron can fire at (within range of frequencies as well)
• Ranges from baseline (no firing) to saturation (maximum firing)

• Threshold Shift upwards, less sensitive
• Temporary - min. hr. days
• Permanent -permanent, deafness (outer, middle, inner, nerve, cortex damage)

• when sounds last too long, too loud, or change to quick, nerves can't react fast enough
• Nerves go to refractory (hyperpolarize)
• nerves permanently damaged/destroyed

Compares detected sound level (standard loudness) across frequencies to determine sensitivity

• Blocking sound from reaching receptors in choclea
• Con-congress-block

• Neural- loss of neurons, nerve, or cortex ability
• Damage to hair cells, nerve fiber, auditory pathway due to age or noise or congenital or tumors or drugs

• Hearing loss due to drugs, noise, or age 
• presbyterian- old man on drugs rocking out
• Worse for higher frequencies

• perception of sound, coninuous or random that is not actually there
• result of sensorinueral damage
• treated with white noise generator

• cerumen blocking canal
• otosclerosis- growth of bone in middle ear blocking ossicle movement
• torn tympanic membrane (ear drum)
• otitis media- middle ear inflammation
• 1: Bypassed by bone conductance to choclea

• cochlea damage where amplitude is too high
• Tears basilar membrane
• destroys tip links in stereocilia
• damages outer hair cells
• Cell death

• excitotoxicity- glutamate flooding in 
• loss of blood flow to cochlea
• free radical damage to tissue and hair cells

• External - detects sound, sends to internal detector (transducer) as electrical signal
• Internal - receives electrical signal, stimulates choclea by going through round window into choclear duct

vestibulochoclear nerve >> choclear nucleus (brain stem)>> trapezoid body (brain stem) and superior olivary nucleus (brain stem) and inferior colliculus (mid brain) >> medial geniculate body (thalamus)

A1 (core)>>rostral core >> rostrotemporal core >> belt (complex sound) >> parabelt (complex sound)

• Anterior portion of core >> prefrontal cortex
• identify sound (decision)

• Posterior core >> belt >> parietal cortex >> prefrontal cortex 
• localize sound

• left and right directions of sound, determined by:
• interaural level diff.
• acoustic shadow
• interaural time diff.
• head motion
• cone of confusion

• medial plane, up and down sound coordinates
• detected by spectral shape cue determined by pinnae

• Distance from head center of sound source
• Determined by perceived loudness:
• blurring effect
• echoes
• doppler effect

• If two sounds, need angle of separation between sources
• 75% correct when azimuth audible angle different by <`10 degrees

• arrival time differences
• differ from 0-600 microsec. depending on azimuth angle

• cone of confusion- sound source equal angle and distance from head, turn head to alter ITD and ILD for each ear
• sound source from 45 degrees has same ItD and ILD as a sound source from 135 degrees straight ahead

• Pinnae alters sound by amplifying and dampening sound waves depending on how elevated the sound source is
• The more complex the sound, the easier to discriminate elevation (more frequencies to work with)

• application of inverse square law on energy of sound wave to determine distance
• greater reduction for higher frequencies

the further away the sound source, the more blurred (less pure) it is, decreases with sound level

• if reflected sound is heard more than direct sound, the source is far away
• if direct sound heard more, source is near

• the sound is louder and high frequency if the source is moving towards you (in front of source)
• softer and lower frequency is moving away (behind source)

• size and shape
• texture (composition)
• elevation
• distance
• azimuth

• Conflicting info from visual (sound from puppet) and auditory (sound from puppeter), have visual dominate by:
• proximity to sound source
• matching movement (timing) with sound source
• match context of sound source

• Neurons tuned for interaural time differences in recieving signal from each ear
• gives info on azimuth of sound source
• Coincidence detectors: only fire if receiving signal from each ear (each choclear nucleus) at same time

• neurons in auditory cortex tuned to detect loudness differences between each ear
• respond with population code to give azimuth info.

• Grouping frequencies together because they have same source or similar sources
• Separate auditory space into streams for analysis

• Onset time similarity
• Location similarity - starting location and follow same path and change slowly (common fate)
• Similarity of timbre and pitch - sound similar

Grouping sounds into auditory stream by sounds have same fundamental frequency and thus same harmonics

onset, change, and offset time of sound need to be similar to mix auditory streams

two diff. frequencies, grouped together when alternating if they have similar frequencies

Two diff. frequencies alternating grouped together if time between alternations is longer

filling in frequency change path when masked by white noise

• Use sound to give info about visual field
• black and white - loudness
• bottom to top - low to high frequency
• left to right- time sound is presented

smallest unit of speech, without meaning, just distinct sound (100 total), compose morpemes, rules of combination

• consists of phonemes, units of language with meaning, composing words
• context- identify root with meaning
• function- identify word with additives (suffixes and prefixes)

sound produced when air is pushed through vocal chords

unrestrict airflow with articulators, producing formants (2-3)

frequencies produced by resonant frequency of vocal cord as air passes over it, determined by airflow determined by articulators

restrict airflow (constriction), shown by rapid formant changes both before and after

overlap of articulation of two phonemes, influence one another

• Depends on size and shape of vocal cords
• Amount of airflow
• size and shape of larynx

frequency and amplitude over time as vowel is produced and frequency and intensity changes

• manner of articulation - how airflow is restricted (stopped, nasal, etc.)
• location of articulation - where airflow is restricted (dental, glottal, etc.)
• voicing - whether or not vocal folds vibrate (voiced) or not (voiceless)

• fixes variability problem where signal doesn't exactly match phonemes because they vary based on:
• context
• coarticulation
• sound producer
• Able to interpret two diff signals as same consonant by probability or completion

top down, assessing whether or not phoneme is likely to appear in the position in a phrase or word

perceptual constancy tool to complete particular missing phonemes by context and top-down (even unconscious)

• consonant followed by vowel
• time between frequencies of consonant end and frequencies of vowel begin
• gap creates phonetic boundary between detected sounds

• compromise perception of phoneme when auditory cues do not match visual cues (blend the two)
• eyes see: ga
• ears hear: ba 
• perceived as: da (midpoint)

• meaning of words 
• primary auditory cortex combined with auditory signal from visual field, communicate with broca and wernickes area

Production speech itself, coordinate speech sound with what needs to be produced, communication between wernickes in auditory cortex to brocas and frontal-motor cortex

Cannot speak, read, or understand words

cannot produce words

cannot understand or contextualize words fluent jargon

dynamics, rhythm, tempo, structure, music theory

• first to last note doubles fundamental frequency
• intervals between 13 notes (separated by percieve frequency) called semitones

• turn- one octave
• chroma - restarts per turn, shows frequency heightening for each tone
• height - determines turn number and thus, octave
• distance- constant between each note, showing equal pitch intervals

variation of loudness throughout peice

• time changes throughout peice
• tempo-overall timing
• beat- pulse describing tempo
• meter- pattern of pulses organized in piece

• how much note (and their harmonics) combinations blend/coincide
• determines dissonance or consonance

knowledge, repetition, familiarity, context , "fit" of notes within scale

speech and timing processing

perception of pitch

music and speech production

• cannot perceive differences in pitch or melody
• congenital
• thicker right inferior frontal cortex and auditory cortex limit connections

difference in time between sound production and first reflection (20 microsec.)

• ratio of low to middle frequencies reflected
• better if lower is reflected more

• ratio of sound received that is reflected versus direct (the more indirect the better)
• the bigger the room, the longer the ideal reverberation time