Back

Modeling Context in Referring Expressions

Licheng YuPatrick PoirsonShan YangAlexander C. BergTamara L. Berg

article2016ECCV1,826 citations

Demonstrates that integrating comparative visual context and joint language generation across same-category objects substantially improves natural language referring expression comprehension and generation on RefCOCO benchmarks.

Listen

Effective human-machine communication requires computational systems to generate and interpret natural language descriptions that unambiguously point out specific objects in visual scenes. This capability is critical for emerging technologies such as interactive robotics and assistive human-computer interfaces. However, existing automated approaches often produce vague or repetitive language because they rely on broad, full-image context rather than comparing the specific visual differences among similar objects in a scene.

The article sets out to demonstrate that explicitly modeling visual comparisons between same-category objects and jointly coordinating language generation across all relevant objects significantly improves both referring expression comprehension and generation.

To evaluate this, the authors developed a deep learning framework combining visual features from deep neural networks with sequential language models. The framework explicitly extracts differences in appearance, location, and relative size between a target item and nearby objects of the same category, while also tying together text generation for all depicted objects of that type. The authors benchmarked their approach across three large-scale datasets comprising over 360,000 referring expressions (RefCOCO, RefCOCO+, and RefCOCOg) using standard automated language metrics, comprehension accuracy benchmarks, human evaluation studies, and an automated object detector.

The findings establish that targeted visual comparisons and joint language generation dramatically outperform generic context baselines. First, incorporating explicit visual difference features boosted referring expression comprehension accuracy across all test sets, improving performance by roughly 4 to 7 percentage points over baseline models. Second, in human evaluations of generated expressions, tying language generation together across same-category objects produced the highest accuracy, outperforming baselines by up to 11 percentage points. Third, joint generation substantially reduced ambiguity, cutting duplicate expression rates—where a model mistakenly produces the exact same phrase for distinct objects in one scene—from 46.27% down to 13.33% on challenging appearance-only benchmarks. Finally, the analysis showed that generic, whole-image context actually degraded comprehension performance, proving that targeted comparisons to same-category objects are the key driver of accuracy.

These results demonstrate that automated agents can generate more natural, precise, and cooperative language by mimicking human conversational strategies. Practically, reducing descriptive ambiguity lowers the risk of task errors and operational failures when autonomous systems or robots interact with users in complex physical settings. Moving forward, teams developing visual-language systems should adopt targeted comparative modeling and joint expression generation over standard global-image context models. Future initiatives should focus on improving automated object detection backends and validating performance in dynamic, real-time interactive dialogues.

The study's primary limitation is that fully automated performance remains heavily dependent on the accuracy of the underlying object detector, which drops when identifying non-human everyday objects in unconstrained scenes. Nonetheless, confidence in the core methodological improvements is high, given consistent performance gains across multiple large datasets and robust validation through human evaluations.

University of North Carolina at Chapel Hillcolor imagesdialoguedigital imagereferring expressionsvisual contextcontext representation
arXiv: 1608.00272lichengunc/refer
Cover for Modeling Context in Referring Expressions

Abstract

Humans refer to objects in their environments all the time, especially in dialogue with other people. We explore generating and comprehending natural language referring expressions for objects in images. In particular, we focus on incorporating better measures of visual context into referring expression models and find that visual comparison to other objects within an image helps improve performance significantly. We also develop methods to tie the language generation process together, so that we generate expressions for all objects of a particular category jointly. Evaluation on three recent datasets - RefCOCO, RefCOCO+, and RefCOCOg, shows the advantages of our methods for both referring expression generation and comprehension.

Topics

color imagesdialoguedigital imagereference imagestatic objectsnatural language generation

Keywords

referring expressionsvisual contextcontext representationexpression comprehensionexpression generationneural image captioningtarget objectLSTM modelconvolutional networks

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Models
  • 3.1 Baselines
  • 3.2 Visual Comparison
  • 3.3 Joint Language Generation
  • 4 Data
  • 4.1 Datasets & Analysis
  • 5 Experiments
  • 5.1 Analysis Experiments
  • Context Representation
  • Visual Comparison
  • 5.2 Referring Expression Comprehension
  • 5.3 Referring Expression Generation
  • 6 Conclusion
  • References

Knowls

  1. Knowl 1 — Visual Comparison Feature Representation for Referring Expressions

    model/method

    To generate discriminative referring expressions for an object in an image containing multiple objects of the same category, a visual comparison representation encodes pairwise differences in appearance, location, and size relative to other objects of the same category.

    For a target object ii, the overall visual representation is constructed from five feature components:

    1. Target object visual features oio_i: extracted from the VGG-16 fc7 layer by forward-passing the target bounding box.

    2. Global image context features gig_i: extracted from the VGG-16 fc7 layer for the full image.

    3. Target bounding box location and size li∈R5l_i \in \mathbb{R}^5: li=[xtlW,ytlH,xbrW,ybrH,w⋅hW⋅H]l_i = \left[ \frac{x_{tl}}{W}, \frac{y_{tl}}{H}, \frac{x_{br}}{W}, \frac{y_{br}}{H}, \frac{w \cdot h}{W \cdot H} \right] where (xtl,ytl)(x_{tl}, y_{tl}) and (xbr,ybr)(x_{br}, y_{br}) are the top-left and bottom-right box coordinates, ww and hh are target box dimensions, and WW and HH are image dimensions.

    4. Visual appearance difference δvi\delta v_i: δvi=1n∑j≠ioi−oj∥oi−oj∥2\delta v_i = \frac{1}{n} \sum_{j \neq i} \frac{o_i - o_j}{\|o_i - o_j\|_2} where nn is the number of comparison objects in the image belonging to the same category as object ii, ojo_j is the VGG-fc7 feature vector of comparison object jj, and differences are normalized and aggregated via average pooling.

    5. Relative location and size difference δli∈R25\delta l_i \in \mathbb{R}^{25}: computed for up to five same-category comparison objects sorted by spatial distance to target ii (zero-padded if fewer than five objects exist). For each comparison object jj, the 5-dimensional relative spatial difference is: δlij=[[Δxtl]ijwi,[Δytl]ijhi,[Δxbr]ijwi,[Δybr]ijhi,wjhjwihi]\delta l_{ij} = \left[ \frac{[\Delta x_{tl}]_{ij}}{w_i}, \frac{[\Delta y_{tl}]_{ij}}{h_i}, \frac{[\Delta x_{br}]_{ij}}{w_i}, \frac{[\Delta y_{br}]_{ij}}{h_i}, \frac{w_j h_j}{w_i h_i} \right] where [Δxtl]ij=xtl,i−xtl,j[\Delta x_{tl}]_{ij} = x_{tl, i} - x_{tl, j}, [Δytl]ij=ytl,i−ytl,j[\Delta y_{tl}]_{ij} = y_{tl, i} - y_{tl, j}, [Δxbr]ij=xbr,i−xbr,j[\Delta x_{br}]_{ij} = x_{br, i} - x_{br, j}, [Δybr]ij=ybr,i−ybr,j[\Delta y_{br}]_{ij} = y_{br, i} - y_{br, j}, and (wi,hi)(w_i, h_i), (wj,hj)(w_j, h_j) are the box dimensions of target ii and comparison object jj.

    The combined representation rir_i is obtained by linear projection of the concatenated features: ri=Wm[oi,gi,li,δvi,δli]+bmr_i = W_m [o_i, g_i, l_i, \delta v_i, \delta l_i] + b_m where WmW_m and bmb_m are learnable weight and bias parameters, and [⋅][\cdot] denotes vector concatenation.

  2. Knowl 2 — Joint Referring Expression Generation via Tied Recurrent Networks

    model/method

    Rather than generating a natural language referring expression for each depicted object independently, joint language generation coordinates the generation across all objects of the same semantic category in an image by tying together the hidden states of Long Short-Term Memory (LSTM) decoders at each time step.

    Given the set of depicted objects of the same category O={oj}O = \{o_j\}, the joint probability of generating referring expressions R={ri}R = \{r_i\} across all objects is: P(R∣O)=∏iP(ri∣oi,{oj≠i},{rj≠i})=∏i∏tP(wit∣wit−1,…,wi1,vi,{hjt,j≠i})P(R \mid O) = \prod_i P(r_i \mid o_i, \{o_{j \neq i}\}, \{r_{j \neq i}\}) = \prod_i \prod_t P(w_{it} \mid w_{it-1}, \dots, w_{i1}, v_i, \{h_{jt, j \neq i}\}) where witw_{it} is the word generated for object ii at time step tt, viv_i is the visual representation of object ii, and hjth_{jt} is the hidden state vector of the LSTM for object jj at time step tt.

    To differentiate expressions and push away ambiguous words, the difference between the hidden output of target object ii and the hidden outputs of all other nn same-category objects is aggregated: hdifit=1n∑j≠ihit−hjt∥hit−hjt∥2h_{\text{dif}_{it}} = \frac{1}{n} \sum_{j \neq i} \frac{h_{it} - h_{jt}}{\|h_{it} - h_{jt}\|_2}

    The probability distribution over the next word is computed by embedding the target hidden state together with the hidden difference vector: P(wit∣wit−1,…,wi1,vi,{hjt,j≠i})=softmax(Wh[hit,hdifit]+bh)P(w_{it} \mid w_{it-1}, \dots, w_{i1}, v_i, \{h_{jt, j \neq i}\}) = \text{softmax}(W_h [h_{it}, h_{\text{dif}_{it}}] + b_h) where WhW_h and bhb_h are trainable projection parameters.

  3. Knowl 3 — People-vs-Objects Dataset Split for Referring Expression Benchmarks

    experimental setup

    In referring expression datasets built upon MSCOCO images (such as RefCOCO and RefCOCO+), roughly half of the target instances belong to the person class. A per-object random partition can place different objects from the same image into both training and test sets, which allows contextual image features to leak across splits.

    To ensure rigorous evaluation of visual context, the people-vs-objects split partitions datasets strictly at the image level:

    • testA: contains test images that depict multiple instances of people.

    • testB: contains test images that depict multiple instances of non-person categories.

    • train: contains remaining images, guaranteeing that all objects from a single image reside exclusively in either the training set or the test set.

  4. Knowl 4 — Impact of Context Scope on Referring Expression Comprehension

    data/table

    Evaluating visual context representations within an LSTM referring expression comprehension baseline demonstrates that incorporating a full-image global feature vector decreases comprehension accuracy relative to utilizing no context features. Conversely, providing localized visual context using scaled bounding boxes centered on the target object consistently enhances comprehension accuracy on RefCOCO and RefCOCO+. Comprehension accuracy corresponds to the percentage of referring expressions where the predicted bounding box achieves IoU>0.5\text{IoU} > 0.5 with the ground-truth annotation.

    Context Source RefCOCO RefCOCO+
    Test A (%) Test B (%) Test A (%) Test B (%)
    No context 63.91 66.31 50.09 45.05
    Global context 63.15 64.21 48.73 42.13
    Scale 2 65.57 67.13 50.38 44.89
    Scale 3 66.14 68.07 50.25 45.40
    Scale 4 66.68 68.56 50.34 45.48

    Here, Scale nn represents visual features extracted from a bounding box scaled by a factor of nn centered on the target object. In addition, computing visual comparisons exclusively against objects of the same category yields higher comprehension accuracy than comparisons against all depicted objects or objects of different categories.

  5. Knowl 5 — Referring Expression Comprehension Performance across Datasets

    data/table

    Referring expression comprehension accuracy (evaluated using the IoU>0.5\text{IoU} > 0.5 criterion with ground truth) across RefCOCO (testA and testB), RefCOCO+ (testA and testB, where spatial location words are disallowed), and RefCOCOg (validation set). Accuracies are reported using ground-truth candidate boxes and automatic detections from Fast R-CNN (denoted with (det)).

    Model RefCOCO RefCOCO+ RefCOCOg
    Test A (%) Test B (%) Test A (%) Test B (%) Validation (%)
    Baseline 63.15 64.21 48.73 42.13 55.16
    visdif 67.57 71.19 52.44 47.51 59.25
    MMI 71.72 71.09 58.42 51.23 62.14
    visdif+MMI 73.98 76.59 59.17 55.62 64.02
    Baseline(det) 58.32 48.48 46.86 34.04 40.75
    visdif(det) 62.50 50.80 50.10 37.48 41.85
    MMI(det) 64.90 54.51 54.03 42.81 45.85
    visdif+MMI(det) 67.64 55.16 55.81 43.43 46.86

    Adding visual comparison features (visdif) produces consistent accuracy gains over baseline models across all dataset splits. The combined model visdif+MMI achieves the best comprehension performance under both ground-truth bounding box and detector-based regimes.

  6. Knowl 6 — Automatic and Human Evaluation of Referring Expression Generation

    data/table

    Referring expression generation models are evaluated using automated metrics (BLEU-1, BLEU-2, ROUGE, METEOR) and human evaluation on RefCOCO, RefCOCO+, and RefCOCOg. In the human evaluation, three annotators click the referred object given the generated sentence and the image; an expression is scored as correct if at least two annotators click the true target.

    Model RefCOCO Human Eval RefCOCO+ Human Eval
    Test A (%) Test B (%) Test A (%) Test B (%)
    Baseline 62.42 64.99 49.18 42.03
    MMI 65.76 68.25 49.84 45.38
    visdif 68.27 74.92 55.20 43.65
    visdif+MMI 70.25 75.47 53.56 47.58
    Baseline+tie 64.51 68.34 52.06 43.53
    visdif+tie 71.40 76.14 57.17 47.92
    visdif+MMI+tie 70.01 76.31 55.64 48.04

    Although Maximum Mutual Information (MMI) training lowers automatic n-gram overlap scores (such as BLEU), human evaluation confirms that MMI training and LSTM tying (+tie) substantially improve communicative precision and resolve ambiguity.

    On the RefCOCOg validation set, automatic generation scores are:

    • Baseline: BLEU-1 0.437, BLEU-2 0.273, ROUGE 0.363, METEOR 0.149
    • MMI: BLEU-1 0.428, BLEU-2 0.263, ROUGE 0.354, METEOR 0.144
    • visdif: BLEU-1 0.442, BLEU-2 0.277, ROUGE 0.370, METEOR 0.151
    • visdif+MMI: BLEU-1 0.430, BLEU-2 0.262, ROUGE 0.356, METEOR 0.145

  7. Knowl 7 — Duplicate Referring Expression Rate in Multi-Object Scenes

    data/table

    The duplicate generation rate is defined as the proportion of images for which an algorithm outputs identical referring expressions for multiple distinct objects within the same image. Lower rates indicate higher uniqueness and discriminability of generated expressions.

    Model RefCOCO RefCOCO+
    Test A (%) Test B (%) Test A (%) Test B (%)
    Baseline 15.60 16.40 28.67 46.27
    MMI 11.60 11.73 21.07 26.40
    visdif 9.20 8.80 19.60 31.07
    visdif+MMI 5.07 6.13 12.13 16.00
    Baseline+tie 11.20 14.93 22.00 32.13
    visdif+tie 4.27 5.33 11.73 16.27
    visdif+MMI+tie 6.53 4.53 10.13 13.33

    Both Maximum Mutual Information (+MMI) training and LSTM tying (+tie) significantly reduce repetitive generation across co-occurring objects. Combining visual comparison, MMI, and language tying (visdif+MMI+tie) achieves the lowest duplication rates across datasets.

  8. Knowl 8 — Sensitivity of Referring Expression Systems to Detector Quality on General Objects

    limitation

    When referring expression comprehension systems are evaluated end-to-end using automated object proposal bounding boxes from Fast R-CNN rather than ground-truth annotations, comprehension accuracy decreases substantially across all models. This degradation is noticeably more pronounced on general non-person object categories (testB) than on people (testA). For instance, under the visdif+MMI model, accuracy on RefCOCO testB drops from 76.59%76.59\% (ground-truth proposals) to 55.16%55.16\% (detector proposals), and on RefCOCO+ testB from 55.62%55.62\% to 43.43%43.43\%, demonstrating that system performance remains heavily constrained by the accuracy of the underlying object detector.

Coverage note — All major contributions—including visual difference feature formulations, tied LSTM joint generation, dataset splits, context scope analysis, comprehension and generation benchmarks, duplicate rate evaluation, and detector limitations—are fully covered.

References

  1. 1.Brown-Schmidt, S., Tanenhaus, M.K.: Watching the eyes when talking about size: an investigation of message formulation and utterance planning. J. Mem. Lang. 54(4), 592–609 (2006)
  2. 2.Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: CVPR (2015)
  3. 3.Fang, H., Gupta, S., Iandola, F., Srivastava, R.K., Deng, L., Doll'ar, P., Gao, J., He, X., Mitchell, M., Platt, J.C., et al.: From captions to visual concepts and back. In: CVPR (2015)
  4. 4.Farhadi, A., Hejrati, M., Sadeghi, M.A., Young, P., Rashtchian, C., Hockenmaier, J., Forsyth, D.: Every picture tells a story: generating sentences from images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 15–29. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15561-1_2
  5. 5.FitzGerald, N., Artzi, Y., Zettlemoyer, L.S.: Learning distributions over logical forms for referring expression generation. In: EMNLP, pp. 1914–1925 (2013)
  6. 6.Funakoshi, K., Watanabe, S., Kuriyama, N., Tokunaga, T.: Generating referring expressions using perceptual groups. In: Belz, A., Evans, R., Piwek, P. (eds.) INLG 2004. LNCS, vol. 3123, pp. 51–60. Springer, Heidelberg (2004)
  7. 7.Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)
  8. 8.Greff, K., Srivastava, R.K., Koutn'ık, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey (2015). arXiv preprint arXiv:1503.04069
  9. 9.Grice, H.P.: Logic and conversation. In: Cole, P., Morgan, J.L. (eds.) Syntax and Semantics: Speech Acts, vol. 3, pp. 41–58. Academic Press, San Diego (1975)
  10. 10.Hodosh, M., Young, P., Hockenmaier, J.: Framing image description as a ranking task: data, models and evaluation metrics. J. Artif. Intell. Res. 47, 853–899 (2013)
  11. 11.Hu, R., Xu, H., Rohrbach, M., Feng, J., Saenko, K., Darrell, T.: Natural language object retrieval. In: CVPR (2016)
  12. 12.Johnson, J., Karpathy, A., Fei-Fei, L.: Densecap: fully convolutional localization networks for dense captioning (2015). arXiv preprint arXiv:1511.07571
  13. 13.Jordan, P., Walker, M.: Learning attribute selections for non-pronominal expressions. In: ACL (2000)
  14. 14.Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: CVPR (2015)
  15. 15.Kazemzadeh, S., Ordonez, V., Matten, M., Berg, T.L.: ReferitGame: referring to objects in photographs of natural scenes. In: EMNLP, pp. 787–798 (2014)
  16. 16.Kelleher, J.D., Kruijff, G.J.M.: Incremental generation of spatial referring expressions in situated dialog. In: ACL (2006)
  17. 17.Kiros, R., Salakhutdinov, R., Zemel, R.S.: Unifying visual-semantic embeddings with multimodal neural language models. In: TACL (2015)
  18. 18.Krahmer, E., Van Deemter, K.: Computational generation of referring expressions: a survey. Comput. Linguist. 38(1), 173–218 (2012)
  19. 19.Kulkarni, G., Premraj, V., Ordonez, V., Dhar, S., Li, S., Choi, Y., Berg, A.C., Berg, T.: Babytalk: understanding and generating simple image descriptions. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2891–2903 (2013)
  20. 20.Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll'ar, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10602-1_48
  21. 21.Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., Murphy, K.: Generation and comprehension of unambiguous object descriptions. In: CVPR (2016)
  22. 22.Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., Yuille, A.: Deep captioning with multimodal recurrent neural networks (m-RNN). In: ICLR (2015)
  23. 23.Mitchell, M., van Deemter, K., Reiter, E.: Natural reference to objects in a visual domain. In: Proceedings of the 6th International Natural Language Generation Conference, pp. 95–104. Association for Computational Linguistics (2010)
  24. 24.Mitchell, M., Reiter, E., van Deemter, K.: Typicality and object reference. Cognitive Science (CogSci) (2013)
  25. 25.Mitchell, M., Van Deemter, K., Reiter, E.: Generating expressions that refer to visible objects. In: HLT-NAACL, pp. 1174–1184 (2013)
  26. 26.Ordonez, V., Kulkarni, G., Berg, T.L.: Im2Text: describing images using 1 million captioned photographs. In: Advances in Neural Information Processing Systems (2011)
  27. 27.Rohrbach, A., Rohrbach, M., Hu, R., Darrell, T., Schiele, B.: Grounding of textual phrases in images by reconstruction (2015). arXiv preprint arXiv:1511.03745
  28. 28.Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
  29. 29.Sadeghi, F., Zitnick, C.L., Farhadi, A.: Visalogy: answering visual analogy questions. In: NIPS (2015)
  30. 30.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint arXiv:1409.1556
  31. 31.Socher, R., Karpathy, A., Le, Q.V., Manning, C.D., Ng, A.Y.: Grounded compositional semantics for finding and describing images with sentences. Trans. Assoc. Comput. Linguist. 2, 207–218 (2014)
  32. 32.Viethen, J., Dale, R.: The use of spatial relations in referring expression generation. In: Proceedings of the Fifth International Natural Language Generation Conference, pp. 59–67. Association for Computational Linguistics (2008)
  33. 33.Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: CVPR (2015)
  34. 34.Winograd, T.: Understanding natural language. Cogn. Psychol. 3(1), 1–191 (1972)
  35. 35.Xu, K., Ba, J., Kiros, R., Courville, A., Salakhutdinov, R., Zemel, R., Bengio, Y.: Show, attend and tell: neural image caption generation with visual attention. In: ICML (2015)

Citation

MLA
Yu, L., et al. “Modeling Context in Referring Expressions”. arXiv, 2016, http://arxiv.org/abs/1608.00272v3.
APA
Yu, L., Poirson, P., Yang, S., Berg, A. C., & Berg, T. L. (2016). Modeling Context in Referring Expressions. arXiv. http://arxiv.org/abs/1608.00272v3
Chicago
Yu, L., P. Poirson, S. Yang, A. C. Berg, and T. L. Berg. 2016. “Modeling Context in Referring Expressions”. arXiv. http://arxiv.org/abs/1608.00272v3.
Harvard
Yu, L. et al. (2016) “Modeling Context in Referring Expressions”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1608.00272v3.
Vancouver
1. Yu L, Poirson P, Yang S, Berg AC, Berg TL (2016) Modeling Context in Referring Expressions. arXiv

BibTeX

@article{yu2016modeling,
  title = {Modeling Context in Referring Expressions},
  author = {Yu, Licheng and Poirson, Patrick and Yang, Shan and Berg, Alexander C. and Berg, Tamara L.},
  year = {2016},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1608.00272v3},
  eprint = {1608.00272}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF