How to concatenate strings and do a find/replace












0















I'm trying to do a find, using regex, and I think this part of my code is fine. This give me two variables: 'findtable' and 'findunzip'. I want to concatenate these two variables into a string, and do multiple replaces in a text file. This is where I have a bug. I think I have to loop through all lines of a text file, line by line, because I can't just do a find/replace all (the variables are different throughout the text file). So, I think my code, below, is close, but something is not quite right. It seems to get into a perpetual loop.



# transform data sets

import glob

import re

path = 'C:\Users\my_path\*.yaml'



find1 = 'steps:.*?fields:'

replace1 = ' global:n   global:n     schema_def:n       fields:'



for fname in glob.glob(path):

    with open(fname, 'r') as f:

        sfile = f.read()



        for line in sfile:



            # start checking line by line

            # get table name

            findtable = 'table_name:s*(.*?)s*id: load'



            # get unzip patterns

            findunzip = 'unzip_patterns:s*(.*?)s*id: extract'



            # drop standardize

            findstd = '- class: pipe.standardize.Standardize.*?id: load'



            concat =   '''steps:

            - id: extract

            conf:

            action_class: pipe.actions.extract.extractor.Extractor

            unzip_patterns:''' + 'n' + findunzip + 'n'

            '''- id: validate

            conf:

            action_class:

            - pipe.actions.preprocess.preprocesser.PreProcesser

            - pipe.actions.validate.validator.Validator

            - id: standardize

            conf:

            action_class: pipe

            .actions.standardize.standardizer.Standardizer

            - id: load

            conf:

            action_class: pipe.actions.load.loader.Loader

            table_name: ''' + findtable



        # done with line by line checking; find/replace all

        text = re.sub(findstd, concat, sfile)

        text = re.sub(find1, replace1, text, flags=re.DOTALL)



        f = open(fname,'w')

        f.write(text)

        f.close()


A sample of the text file that I am working with:



        id: standardize

      - class: pipe.steps.load.Load

        conf:

          table_name: snl_realestate_hb_na_fundamentals1_o

        id: load

    - id: snl_realestate_hb_na_totalreturn

      steps:



      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Period

            - data_type: FLOAT



    - id: snl_realestate_hb_na_volume

      steps:

      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Ticker



    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_fundamentals1_o

    id: load

- id: snl_realestate_hb_na_totalreturn

  steps:



  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT



- id: snl_realestate_hb_na_volume

  steps:

  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Ticker



  - class: pipe.steps.standardize.Standardize

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT

          name: Total_Return

        - data_type: FLOAT

          name: TR_SNL_Broad_Index_Change

        - data_type: FLOAT

          name: TR_SandP_500

        - data_type: DATETIME

          name: Beginning_Pricing_Date

    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_totalreturn

    id: load



    #######


So, the 'concat' string would ultimately look like this:



  steps:

  - id: extract

    conf:

      action_class: pipe.actions.extract.extractor.Extractor

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

  - id: validate

    conf:

      action_class:

      - pipe.actions.preprocess.preprocesser.PreProcesser

      - pipe.actions.validate.validator.Validator

  - id: standardize

    conf:

      action_class: pipe.actions.standardize.standardizer.Standardizer

  - id: load

    conf:

      action_class: pipe.actions.load.loader.Loader

      table_name: snl_realestate_hb_na_fundamentals1_o


There are several 'concat' strings that I want to replace in each text file. At the end of the line-by-line checking, I want to do a final find/replace, and this is the same thing for the entire file, so I think I can get this all done in one go. I added comments to make the logic more readable.






python regex python-3.x







share|improve this question




















  • 3





    Please provide a minimal input file on which this exhibits the problem.

    – Prune
    Nov 15 '18 at 21:40











  • In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

    – sln
    Nov 15 '18 at 21:49













  • I just added some sample text. The actual text file is huge.

    – ryguy72
    Nov 15 '18 at 21:57






  • 1





    Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

    – Anthon
    Nov 16 '18 at 6:15











  • I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

    – ryguy72
    Nov 16 '18 at 15:07
















0















I'm trying to do a find, using regex, and I think this part of my code is fine. This give me two variables: 'findtable' and 'findunzip'. I want to concatenate these two variables into a string, and do multiple replaces in a text file. This is where I have a bug. I think I have to loop through all lines of a text file, line by line, because I can't just do a find/replace all (the variables are different throughout the text file). So, I think my code, below, is close, but something is not quite right. It seems to get into a perpetual loop.



# transform data sets

import glob

import re

path = 'C:\Users\my_path\*.yaml'



find1 = 'steps:.*?fields:'

replace1 = ' global:n   global:n     schema_def:n       fields:'



for fname in glob.glob(path):

    with open(fname, 'r') as f:

        sfile = f.read()



        for line in sfile:



            # start checking line by line

            # get table name

            findtable = 'table_name:s*(.*?)s*id: load'



            # get unzip patterns

            findunzip = 'unzip_patterns:s*(.*?)s*id: extract'



            # drop standardize

            findstd = '- class: pipe.standardize.Standardize.*?id: load'



            concat =   '''steps:

            - id: extract

            conf:

            action_class: pipe.actions.extract.extractor.Extractor

            unzip_patterns:''' + 'n' + findunzip + 'n'

            '''- id: validate

            conf:

            action_class:

            - pipe.actions.preprocess.preprocesser.PreProcesser

            - pipe.actions.validate.validator.Validator

            - id: standardize

            conf:

            action_class: pipe

            .actions.standardize.standardizer.Standardizer

            - id: load

            conf:

            action_class: pipe.actions.load.loader.Loader

            table_name: ''' + findtable



        # done with line by line checking; find/replace all

        text = re.sub(findstd, concat, sfile)

        text = re.sub(find1, replace1, text, flags=re.DOTALL)



        f = open(fname,'w')

        f.write(text)

        f.close()


A sample of the text file that I am working with:



        id: standardize

      - class: pipe.steps.load.Load

        conf:

          table_name: snl_realestate_hb_na_fundamentals1_o

        id: load

    - id: snl_realestate_hb_na_totalreturn

      steps:



      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Period

            - data_type: FLOAT



    - id: snl_realestate_hb_na_volume

      steps:

      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Ticker



    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_fundamentals1_o

    id: load

- id: snl_realestate_hb_na_totalreturn

  steps:



  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT



- id: snl_realestate_hb_na_volume

  steps:

  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Ticker



  - class: pipe.steps.standardize.Standardize

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT

          name: Total_Return

        - data_type: FLOAT

          name: TR_SNL_Broad_Index_Change

        - data_type: FLOAT

          name: TR_SandP_500

        - data_type: DATETIME

          name: Beginning_Pricing_Date

    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_totalreturn

    id: load



    #######


So, the 'concat' string would ultimately look like this:



  steps:

  - id: extract

    conf:

      action_class: pipe.actions.extract.extractor.Extractor

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

  - id: validate

    conf:

      action_class:

      - pipe.actions.preprocess.preprocesser.PreProcesser

      - pipe.actions.validate.validator.Validator

  - id: standardize

    conf:

      action_class: pipe.actions.standardize.standardizer.Standardizer

  - id: load

    conf:

      action_class: pipe.actions.load.loader.Loader

      table_name: snl_realestate_hb_na_fundamentals1_o


There are several 'concat' strings that I want to replace in each text file. At the end of the line-by-line checking, I want to do a final find/replace, and this is the same thing for the entire file, so I think I can get this all done in one go. I added comments to make the logic more readable.






python regex python-3.x







share|improve this question




















  • 3





    Please provide a minimal input file on which this exhibits the problem.

    – Prune
    Nov 15 '18 at 21:40











  • In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

    – sln
    Nov 15 '18 at 21:49













  • I just added some sample text. The actual text file is huge.

    – ryguy72
    Nov 15 '18 at 21:57






  • 1





    Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

    – Anthon
    Nov 16 '18 at 6:15











  • I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

    – ryguy72
    Nov 16 '18 at 15:07














0












0








0








I'm trying to do a find, using regex, and I think this part of my code is fine. This give me two variables: 'findtable' and 'findunzip'. I want to concatenate these two variables into a string, and do multiple replaces in a text file. This is where I have a bug. I think I have to loop through all lines of a text file, line by line, because I can't just do a find/replace all (the variables are different throughout the text file). So, I think my code, below, is close, but something is not quite right. It seems to get into a perpetual loop.



# transform data sets

import glob

import re

path = 'C:\Users\my_path\*.yaml'



find1 = 'steps:.*?fields:'

replace1 = ' global:n   global:n     schema_def:n       fields:'



for fname in glob.glob(path):

    with open(fname, 'r') as f:

        sfile = f.read()



        for line in sfile:



            # start checking line by line

            # get table name

            findtable = 'table_name:s*(.*?)s*id: load'



            # get unzip patterns

            findunzip = 'unzip_patterns:s*(.*?)s*id: extract'



            # drop standardize

            findstd = '- class: pipe.standardize.Standardize.*?id: load'



            concat =   '''steps:

            - id: extract

            conf:

            action_class: pipe.actions.extract.extractor.Extractor

            unzip_patterns:''' + 'n' + findunzip + 'n'

            '''- id: validate

            conf:

            action_class:

            - pipe.actions.preprocess.preprocesser.PreProcesser

            - pipe.actions.validate.validator.Validator

            - id: standardize

            conf:

            action_class: pipe

            .actions.standardize.standardizer.Standardizer

            - id: load

            conf:

            action_class: pipe.actions.load.loader.Loader

            table_name: ''' + findtable



        # done with line by line checking; find/replace all

        text = re.sub(findstd, concat, sfile)

        text = re.sub(find1, replace1, text, flags=re.DOTALL)



        f = open(fname,'w')

        f.write(text)

        f.close()


A sample of the text file that I am working with:



        id: standardize

      - class: pipe.steps.load.Load

        conf:

          table_name: snl_realestate_hb_na_fundamentals1_o

        id: load

    - id: snl_realestate_hb_na_totalreturn

      steps:



      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Period

            - data_type: FLOAT



    - id: snl_realestate_hb_na_volume

      steps:

      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Ticker



    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_fundamentals1_o

    id: load

- id: snl_realestate_hb_na_totalreturn

  steps:



  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT



- id: snl_realestate_hb_na_volume

  steps:

  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Ticker



  - class: pipe.steps.standardize.Standardize

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT

          name: Total_Return

        - data_type: FLOAT

          name: TR_SNL_Broad_Index_Change

        - data_type: FLOAT

          name: TR_SandP_500

        - data_type: DATETIME

          name: Beginning_Pricing_Date

    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_totalreturn

    id: load



    #######


So, the 'concat' string would ultimately look like this:



  steps:

  - id: extract

    conf:

      action_class: pipe.actions.extract.extractor.Extractor

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

  - id: validate

    conf:

      action_class:

      - pipe.actions.preprocess.preprocesser.PreProcesser

      - pipe.actions.validate.validator.Validator

  - id: standardize

    conf:

      action_class: pipe.actions.standardize.standardizer.Standardizer

  - id: load

    conf:

      action_class: pipe.actions.load.loader.Loader

      table_name: snl_realestate_hb_na_fundamentals1_o


There are several 'concat' strings that I want to replace in each text file. At the end of the line-by-line checking, I want to do a final find/replace, and this is the same thing for the entire file, so I think I can get this all done in one go. I added comments to make the logic more readable.






python regex python-3.x







share|improve this question
















I'm trying to do a find, using regex, and I think this part of my code is fine. This give me two variables: 'findtable' and 'findunzip'. I want to concatenate these two variables into a string, and do multiple replaces in a text file. This is where I have a bug. I think I have to loop through all lines of a text file, line by line, because I can't just do a find/replace all (the variables are different throughout the text file). So, I think my code, below, is close, but something is not quite right. It seems to get into a perpetual loop.



# transform data sets

import glob

import re

path = 'C:\Users\my_path\*.yaml'



find1 = 'steps:.*?fields:'

replace1 = ' global:n   global:n     schema_def:n       fields:'



for fname in glob.glob(path):

    with open(fname, 'r') as f:

        sfile = f.read()



        for line in sfile:



            # start checking line by line

            # get table name

            findtable = 'table_name:s*(.*?)s*id: load'



            # get unzip patterns

            findunzip = 'unzip_patterns:s*(.*?)s*id: extract'



            # drop standardize

            findstd = '- class: pipe.standardize.Standardize.*?id: load'



            concat =   '''steps:

            - id: extract

            conf:

            action_class: pipe.actions.extract.extractor.Extractor

            unzip_patterns:''' + 'n' + findunzip + 'n'

            '''- id: validate

            conf:

            action_class:

            - pipe.actions.preprocess.preprocesser.PreProcesser

            - pipe.actions.validate.validator.Validator

            - id: standardize

            conf:

            action_class: pipe

            .actions.standardize.standardizer.Standardizer

            - id: load

            conf:

            action_class: pipe.actions.load.loader.Loader

            table_name: ''' + findtable



        # done with line by line checking; find/replace all

        text = re.sub(findstd, concat, sfile)

        text = re.sub(find1, replace1, text, flags=re.DOTALL)



        f = open(fname,'w')

        f.write(text)

        f.close()


A sample of the text file that I am working with:



        id: standardize

      - class: pipe.steps.load.Load

        conf:

          table_name: snl_realestate_hb_na_fundamentals1_o

        id: load

    - id: snl_realestate_hb_na_totalreturn

      steps:



      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Period

            - data_type: FLOAT



    - id: snl_realestate_hb_na_volume

      steps:

      - class: pipe.steps.extract.Extract

        conf:

          unzip_patterns:

          - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

        id: extract

      - class: pipe.steps.validate.Validate

        conf:

          schema_def:

            fields:

            - data_type: STRING

              name: Operation

            - data_type: STRING

              name: SNL_Institution_Key

            - data_type: STRING

              name: Institution_Name

            - data_type: STRING

              name: Ticker



    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_fundamentals1_o

    id: load

- id: snl_realestate_hb_na_totalreturn

  steps:



  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_TotalReturn.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT



- id: snl_realestate_hb_na_volume

  steps:

  - class: pipe.steps.extract.Extract

    conf:

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

    id: extract

  - class: pipe.steps.validate.Validate

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Ticker



  - class: pipe.steps.standardize.Standardize

    conf:

      schema_def:

        fields:

        - data_type: STRING

          name: Operation

        - data_type: STRING

          name: SNL_Institution_Key

        - data_type: STRING

          name: Institution_Name

        - data_type: STRING

          name: Period

        - data_type: FLOAT

          name: Total_Return

        - data_type: FLOAT

          name: TR_SNL_Broad_Index_Change

        - data_type: FLOAT

          name: TR_SandP_500

        - data_type: DATETIME

          name: Beginning_Pricing_Date

    id: standardize

  - class: pipe.steps.load.Load

    conf:

      table_name: snl_realestate_hb_na_totalreturn

    id: load



    #######


So, the 'concat' string would ultimately look like this:



  steps:

  - id: extract

    conf:

      action_class: pipe.actions.extract.extractor.Extractor

      unzip_patterns:

      - .*RealEstate_Volume.*_{FD_YYYYMMDD}.*

  - id: validate

    conf:

      action_class:

      - pipe.actions.preprocess.preprocesser.PreProcesser

      - pipe.actions.validate.validator.Validator

  - id: standardize

    conf:

      action_class: pipe.actions.standardize.standardizer.Standardizer

  - id: load

    conf:

      action_class: pipe.actions.load.loader.Loader

      table_name: snl_realestate_hb_na_fundamentals1_o


There are several 'concat' strings that I want to replace in each text file. At the end of the line-by-line checking, I want to do a final find/replace, and this is the same thing for the entire file, so I think I can get this all done in one go. I added comments to make the logic more readable.






python regex python-3.x




python regex python-3.x






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 16 '18 at 6:09









Anthon

29k1693145




29k1693145










asked Nov 15 '18 at 21:37









ryguy72ryguy72

4,0731719




4,0731719








  • 3





    Please provide a minimal input file on which this exhibits the problem.

    – Prune
    Nov 15 '18 at 21:40











  • In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

    – sln
    Nov 15 '18 at 21:49













  • I just added some sample text. The actual text file is huge.

    – ryguy72
    Nov 15 '18 at 21:57






  • 1





    Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

    – Anthon
    Nov 16 '18 at 6:15











  • I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

    – ryguy72
    Nov 16 '18 at 15:07














  • 3





    Please provide a minimal input file on which this exhibits the problem.

    – Prune
    Nov 15 '18 at 21:40











  • In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

    – sln
    Nov 15 '18 at 21:49













  • I just added some sample text. The actual text file is huge.

    – ryguy72
    Nov 15 '18 at 21:57






  • 1





    Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

    – Anthon
    Nov 16 '18 at 6:15











  • I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

    – ryguy72
    Nov 16 '18 at 15:07








3




3





Please provide a minimal input file on which this exhibits the problem.

– Prune
Nov 15 '18 at 21:40





Please provide a minimal input file on which this exhibits the problem.

– Prune
Nov 15 '18 at 21:40













In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

– sln
Nov 15 '18 at 21:49







In Part-A, loop through the lines gathering parts for the replace. Construct the replacement string when done. In Part-B, loop through the lines using the alternations (regex) of parts you want to replace. Replace with the constructed replace string. It has to be done in two steps.

– sln
Nov 15 '18 at 21:49















I just added some sample text. The actual text file is huge.

– ryguy72
Nov 15 '18 at 21:57





I just added some sample text. The actual text file is huge.

– ryguy72
Nov 15 '18 at 21:57




1




1





Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

– Anthon
Nov 16 '18 at 6:15





Since sfile is a string, the for line in sfile will loop over the characters in the string. You probably want to do something like for line in sfile.splitlines(True). On the other hand it is always a bad idea to try and parse YAML using regex. There are so many ways the input might be subtle different (indent, flow/block-style, quoted/unquoted scalars, comments) but semantically the same that are easy for the parser but impossible for a (maintainable) regex to process.

– Anthon
Nov 16 '18 at 6:15













I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

– ryguy72
Nov 16 '18 at 15:07





I still have a slight problem with this code, but that definitely helped! Thanks for the tip!!

– ryguy72
Nov 16 '18 at 15:07












0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53328247%2fhow-to-concatenate-strings-and-do-a-find-replace%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53328247%2fhow-to-concatenate-strings-and-do-a-find-replace%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Guess what letter conforming each word

Image
6 $begingroup$ This is an entry in the Fortnightly Topic Challenge #41: Short and Sweet. 1 letter only could be mean anything . With an additional letter in front , could be used as medicine component and something you definitely learn or at least heard once. Add another letter in front , you would possibly hear that many times . Add an additional letter in front , you would found an independent country . With an additional letter, in the end , you would found something related to apple . Add another letter in the end , you would found something bad . Hint: Each letter is unique . word knowledge letters chemistry share | improve this question ...
Read more

Port of Spain

Image
For ports in the country of Spain, see List of ports in Spain. City in City of Port of Spain, Trinidad and Tobago Port of Spain City City of Port of Spain Top to bottom (left to right): Skyline of Port of Spain from Gulf of Paria; National Academy for the Performing Arts, Independence Square, Eric Williams Plaza; MovieTowne, Port of Port of Spain Motto(s):  "We Gather Strength As We Go Along" Port of Spain Coordinates: 10°40′N 61°31′W  /  10.667°N 61.517°W  / 10.667; -61.517 Coordinates: 10°40′N 61°31′W  /  10.667°N 61.517°W  / 10.667; -61.517 Country   Trinidad and Tobago Jurisdiction City of Port of Spain Settled 1560 Incorporated (town) 1853 Incorporated (city) 1914 Government  • Mayor Joel Martinez  • Governing body City Corporation Area [1]  • City 12 km 2 (5 sq mi) Elevation [2] 66 m (217 ft) Population (2011) [1]  • City 37,074  • Rank 3rd  • Density 3,090/km 2 (8,000/sq mi)...
Read more

Run scheduled task as local user group (not BUILTIN)

Image
0 I am trying to run a scheduled task with User Group (not a single user). This is working fine when I in the field "when running a task, use the following user account:" insert BUILTIN User Group (for example: Administrators or Users). But when I try it with the created Local User Group (example: TestGroup) it is not working. The Users in the TestGroup are the same as in the Administrators group. What could be the problem? Are there some restrictions to the Created User Group compared to BUILTINUsers? scheduled-tasks taskscheduler windowstaskschedule share | improve this question asked Nov 15 '18 at 15:36 Far...
Read more