r - creating corpus from multiple html text files - Stack Overflow
Description: I have a list of html files, I have taken some texts from the web and make them read with the read_html. My files names are like: a1 &lt;- read_html(link of the text) a2 &lt;- read_html(link o...

# creating corpus from multiple html text files

Ask Question
Asked 7 years, 1 month ago
Modified 7 years, 1 month ago
Viewed 473 times
Part of R Language Collective
0

I have a list of html files, I have taken some texts from the web and make them read with the `read_html`.

My files names are like:

    a1 <- read_html(link of the text)
    a2 <- read_html(link of the text)
    .
    .
    . ## until:
    a100 <- read_html(link of the text)

I am trying to create a corpus with these.

Any ideas how can I do it?

Thanks.

- r
- text
- corpus

edited Nov 20, 2018 at 10:21
jogo
12.6k1111 gold badges4141 silver badges4343 bronze badges
asked Nov 20, 2018 at 10:20
SBA
29911 gold badge22 silver badges1212 bronze badges
2

- 2
  Use a list `L <- lapply(vector_of_links, read_html)`
  jogo
  – jogo
  2018-11-20 10:22:58 +00:00
  Commented Nov 20, 2018 at 10:22
- Thanks. Do you think that it is a true way to create the vector of the links as following `v<-c(a(1:100))`
  SBA
  – SBA
  2018-11-20 10:26:07 +00:00
  Commented Nov 20, 2018 at 10:26

Add a comment |

## 2 Answers 2

Sorted by: Reset to default
0

You could allocate the vector beforehand:

    text <- rep(NA, 100)
    text[1] <- read_html(link1)
    ...
    text[100] <- read_html(link100)

Even better, if you organize your links as vector. Then you can use, as suggested in the comments, `lapply`:

    text <- lapply(links, read_html)

(here links is a vector of the links).

It would be rather bad coding style to use `assign`:

    # not a good idea
    for (i in 1:100) assign(paste0("text", i), get(paste0("link", i)))

since this is rather slow and hard to process further.

edited Nov 30, 2018 at 13:05
answered Nov 20, 2018 at 12:03
Karsten W.
18.7k1212 gold badges7474 silver badges114114 bronze badges
Sign up to request clarification or add additional context in comments.

## Comments

0

I would suggest using `purrr` for this solution:

    library(tidyverse)
    library(purrr)
    library(rvest)

    files <- list.files("path/to/html_links", full.names = T)

    all_html <- tibble(file_path = files) %>%
      mutate(filenames = basename(files)) %>%
      mutate(text = map(file_path, read_html))

Is a nice way to keep track of which piece of text belongs to which file. It also makes things like sentiment or any other type analysis easy at a document level.

answered Nov 20, 2018 at 12:44
Hanjo Odendaal
1,47122 gold badges1414 silver badges3232 bronze badges

## Comments

## Your Answer

Thanks for contributing an answer to Stack Overflow!

- Please be sure to answer the question. Provide details and share your research!

But avoid …

- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Draft saved
Draft discarded

### Sign up or log in

Sign up using Google
Sign up using Email and Password

### Post as a guest

Name
Email

Required, but never shown

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

- r
- text
- corpus

See similar questions with these tags.

lang-r